diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 4ea22141a..000000000 --- a/.travis.yml +++ /dev/null @@ -1,25 +0,0 @@ -language: python -python: - - 3.4 -notifications: - email: false - slack: etsnlp:TLmcZ7JGkPMv5eEpGxUuFm5k - -# run on the new Travis infrastructure -sudo: false - -# Install stuff -before_install: - - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh - - chmod +x miniconda.sh - - ./miniconda.sh -b - - export PATH=/home/travis/miniconda3/bin:$PATH - - conda config --add channels desilinguist - - conda update --yes conda -install: - - conda install --file conda_requirements.txt --yes - - pip install -e . - -# Run test -script: - - nosetests -v tests/test_experiment.py:test_run_experiment_lr diff --git a/README.md b/README.md index b04ea36d6..2baf02b75 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ ## Introduction -RSMTool is a python package for facilitating research on building and evaluating scoring models (SMs) for automated scoring engines. It allows the integration of educational measurement practices with the automated scoring and model building process. +RSMTool is a python package for facilitating research on building and evaluating scoring models (SMs) for automated scoring engines. It allows the integration of educational measurement practices with the automated scoring and model building process. See [rsmtool.pdf](doc/rsmtool.pdf) for background information. Specifically, RSMTool takes a feature file with numeric, non-sparse features and a human score as input and lets you try several different regression models to try and predict the human score from the features. The primary output of RSMTool is a comprehensive, customizable HTML statistical report that contains feature descriptives, subgroup analyses, model statistics, as well as several different evaluation measures illustrating model efficacy. The various numbers and figures in the report are highlighted based on whether they exceed or fall short of the recommendations laid out by Williamson et al. (2012). However, these can be easily customized if the user wishes to use different set of recommendations. -Finally, since the report is based on IPython notebooks, it can be easily customized. In addition, RSMTool explicitly provides support for adding custom notebooks to the report. [Here's](http://bit.ly/rsmtool) an example RSMTool report for a simple scoring system built to automatically score the responses from the [2012 Kaggle Automated Student Assessment Prize competition](https://www.kaggle.com/c/asap-aes). +Finally, since the report is based on IPython notebooks, it can be easily customized. In addition, RSMTool explicitly provides support for adding custom notebooks to the report. RSMTool provides the following main scripts: @@ -25,10 +25,6 @@ David M. Williamson, Xiaoming Xi, and F. Jay Breyer. 2012. A Framework for Evalu ## Installation -If you want to use RSMTool on your own machine, either as a user or a developer, follow the appropriate instructions below. Note that RSMTool only works with Python 3.4 and higher. - -### For users - Currently, the best way to install RSMTool is by using the `conda` package manager. If you have the `conda` package manager already installed, you can skip straight to Step 2. 1. To install the `conda` package manager, follow the instructions on [this page](http://conda.pydata.org/docs/install/quick.html). @@ -39,25 +35,36 @@ Currently, the best way to install RSMTool is by using the `conda` package manag 4. From now on, you will need to activate this conda environment whenever you want to use RSMTool. This will ensure that the packages required by `rsmtool` will only be used when you want to run `rsmtool` experiments and will not affect other projects. -### For developers +Note that RSMTool only works with Python 3.4 and higher. + +## Example + +You can try out RSMTool as follows: + +1. Go to the `example` folder. This folder contains the training and test set features for a simple scoring system built to automatically score the responses from the [2012 Kaggle Automated Student Assessment Prize competition](https://www.kaggle.com/c/asap-aes). +2. Make sure to activate the conda environment where you installed rsmtool (e.g., `source activate rsmtool`) +3. Run RSMTool: `rsmtool config.json` +4. Since no output directory was specfied, `rsmtool` will create the three output folders in the current directory: `figure`, `output`, and `report`. You can examine the HTML report `report/ASAP2_report.html`. It should look like [this](https://s3.amazonaws.com/sample-rsmtool-report/ASAP2_report.html). + +## Contributing -The instructions below are only if you are developing new features or functionality for RSMTool. +Contributions to RSMTool are very welcome. You can use the instructions below to get started on developing new features or functionality for RSMTool. -1. Pull the latest version of rsmtool from github and switch to the develop branch. +1. Pull the latest version of rsmtool from github and switch to the `master` branch. 2. If you already have the `conda` package manager installed, skip to the next step. If you do not, follow the instructions on [this page](http://conda.pydata.org/docs/install/quick.html) to install `conda`. -3. Create a new conda environment (say, `rsmtool`) and install the packages specified in the `conda_requirements.txt` file by running `conda create -n rsmtool -c desilinguist --file conda_requirements.txt`. Use `conda_requirements_windows.txt` if you are on Windows. The two conda requirements file will be consolidated with the next version. +3. Create a new conda environment (say, `rsmtool`) and install the packages specified in the `conda_requirements.txt` file by running `conda create -n rsmtool -c desilinguist --file conda_requirements.txt`. Use `conda_requirements_windows.txt` if you are on Windows. There are two versions because RSMTool currently does not use MKL on non-Windows platforms. 4. Activate the environment using `source activate rsmtool` (use `activate rsmtool` if you are on Windows). 5. Run `pip install -e .` to install rsmtool into the environment in editable mode which is what we need for development. -6. Run `nosetests -v tests` to run the tests. +6. Run `nosetests -v tests` to run the tests. ## Available documentation -## Usage documentation for main scripts +### Usage documentation for main scripts * [rsmtool](doc/rsmtool.md) @@ -67,7 +74,7 @@ The instructions below are only if you are developing new features or functional * [rsmcompare](doc/rsmcompare.md) -## Description of configuration files +### Description of configuration files * [RSMTool configuration file](doc/config_file.md) - main configuration file for `rsmtool` @@ -79,7 +86,7 @@ The instructions below are only if you are developing new features or functional * [Feature file](doc/feature_file.md) - feature file -## Lists of available options +### Lists of available options * [Available models](doc/available_models.md) - list of models available to `rsmtool` @@ -89,7 +96,7 @@ The instructions below are only if you are developing new features or functional * [Output CSV files](doc/output_csv.md) - .csv files generated by `rsmtool` and `rsmeval` -## Documentation for developers +### Documentation for developers * [New notebooks](doc/new_notebooks.md) - the variables and data frames available for use in custom report sections. diff --git a/conda-recipe/unix/rsmtool/meta.yaml b/conda-recipe/unix/rsmtool/meta.yaml index 6135e1873..553f7190c 100644 --- a/conda-recipe/unix/rsmtool/meta.yaml +++ b/conda-recipe/unix/rsmtool/meta.yaml @@ -1,6 +1,6 @@ package: name: rsmtool - version: 5.0.1 + version: 5.0.2 source: path: ../../../../rsmtool diff --git a/conda-recipe/windows/rsmtool/meta.yaml b/conda-recipe/windows/rsmtool/meta.yaml index 635cf9c2f..bb78041aa 100644 --- a/conda-recipe/windows/rsmtool/meta.yaml +++ b/conda-recipe/windows/rsmtool/meta.yaml @@ -1,6 +1,6 @@ package: name: rsmtool - version: 5.0.1 + version: 5.0.2 source: path: ../../../../rsmtool diff --git a/doc/CHANGELOG.md b/doc/CHANGELOG.md index 1d0094bd1..c940023c9 100644 --- a/doc/CHANGELOG.md +++ b/doc/CHANGELOG.md @@ -1,5 +1,9 @@ # Release Notes +## v5.0.2 (June 27, 2016) + +Added files needed for [Journal of Open Source Software](http://joss.theoj.org/) submission. + ## v5.0.1 (June 7, 2016) ### Bugfixes diff --git a/doc/new_notebooks.md b/doc/new_notebooks.md index d5c8af280..57cfe9d24 100644 --- a/doc/new_notebooks.md +++ b/doc/new_notebooks.md @@ -1,6 +1,6 @@ # Guide to writing new IPython notebooks -`rsmtool` allows developers and users to contribute new analysis sections to `rsmtool` and `rsmeval` as *custom sections* (see report_sections.md for further information). +RSMTool allows developers and users to contribute new analysis sections to `rsmtool` and `rsmeval` as *custom sections* (see report_sections.md for further information). When writing such notebooks, some or all of the python variables below will be available in the notebook and so can be used in the notebooks. @@ -44,20 +44,17 @@ When writing such notebooks, some or all of the python variables below will be a - `df_train_metadata` and `df_test_metadata`: Data frames containing the `*_train_metadata.csv` and `*_test_metadata.csv` files respectively as explained in `doc/output_csv.md`. [`rsmtool`: both data frames, `rsmeval`: test data only] - - `df_train_length`: A data frame containing `spkitemid` and response lengths (`length`) for the training data. These are *only* available (a) if `length_column` was specified in the config file, (b) if no values in that column are missing and, (c) if the values in that column are not distributed with a standard deviation <= 0. [`rsmtool` only] - `df_test_human_scores`: A data frame containing `spkitemid`, test label (`sc1`) and the second human score (`sc2`) for the test data. This frame is *only* available if `second_human_score_column` was specified in the config file. Note that the data frame will contain `NaN`s for the responses for which no numeric second human score was available or for which the second score was 0 and exclude_zero_scores was set to `True`. - `df_pred_preproc`: A data frame containing the `*_pred_processed.csv` file as explained in `doc/output_csv.md`. -- `df_feature_subset_specs`: a data frame containing the content of `feature_subset_file` if it was specified in config file. `None` if not specified in the config file. -[`rsmtool` only] +- `df_feature_subset_specs`: a data frame containing the content of `feature_subset_file` if it was specified in config file. `None` if not specified in the config file. [`rsmtool` only] In addition, the following variables are also available but you should *not* re-read the files under these directories which are already available as data frames. - - `output_dir`: The output directory for the experiment that contains all the generated CSV files. -- + - `figure_dir`: The figure directory for the experiment that contains all the generated SVG and PNG figures. ## Notes: diff --git a/doc/rsmtool.pdf b/doc/rsmtool.pdf new file mode 100644 index 000000000..d21eb64dc Binary files /dev/null and b/doc/rsmtool.pdf differ diff --git a/example/config.json b/example/config.json new file mode 100644 index 000000000..a262696c6 --- /dev/null +++ b/example/config.json @@ -0,0 +1,16 @@ +{ + "test_label_column": "score", + "train_file": "train.csv", + "description": "Using all features with a LinearRegression model.", + "use_scaled_predictions": true, + "trim_min": 1, + "id_column": "ID", + "model": "LinearRegression", + "train_label_column": "score", + "second_human_score_column": "score2", + "length_column": "LENGTH", + "features": "features.json", + "experiment_id": "ASAP2", + "trim_max": 6, + "test_file": "test.csv" +} diff --git a/example/features.json b/example/features.json new file mode 100644 index 000000000..2db158756 --- /dev/null +++ b/example/features.json @@ -0,0 +1,44 @@ +{ + "features": [ + { + "transform": "raw", + "feature": "FEATURE1", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE2", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE3", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE4", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE5", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE6", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE7", + "sign": 1 + }, + { + "transform": "raw", + "feature": "FEATURE8", + "sign": 1 + } + ] +} diff --git a/example/test.csv b/example/test.csv new file mode 100644 index 000000000..d266fd15b --- /dev/null +++ b/example/test.csv @@ -0,0 +1,201 @@ +ID,FEATURE1,FEATURE2,FEATURE3,FEATURE4,FEATURE5,FEATURE6,FEATURE7,FEATURE8,LENGTH,QUESTION,L1,score,score2 +RESPONSE_1,5.82894561761021,-0.112722462879731,-0.19553847221876106,0.0,-0.127186754767292,4.61176470588235,12,-6480.182774817528,680,QUESTION_1,Vulcan,4,4 +RESPONSE_2,5.785362461648622,-0.0860865371140256,-0.103695169473043,0.0,-0.0678844233302131,4.533026113671268,20,-4929.680684216342,651,QUESTION_1,Esperanto,4,4 +RESPONSE_3,4.67749084756772,-0.113823323306676,-0.18043874177926295,-0.272797735788189,-0.0964485644340824,4.646511627906982,4,-5075.591346790763,215,QUESTION_1,Vulcan,4,3 +RESPONSE_4,5.14166355650266,-0.0858582517594522,-0.120912708351669,-0.24182541670333704,-0.0764719112901873,4.684210526315789,1,-12275.301536590043,342,QUESTION_1,Navi,3,4 +RESPONSE_5,5.30081424674662,-0.0459878517494697,-0.149812850831677,-0.303758665198693,-0.122321680483099,4.24937655860349,7,-10578.234758280234,401,QUESTION_1,Vulcan,2,3 +RESPONSE_6,4.26969744969996,-0.149841869223137,-0.27735009811261496,-0.409673245199351,-0.0836242010007091,3.88111888111888,17,-7035.4431976620635,143,QUESTION_1,Klingon,2,2 +RESPONSE_7,6.2971093199339405,-0.105956834438511,-0.28140662203383504,0.0,-0.121379425806881,4.2486187845303895,5,-9650.865430119216,543,QUESTION_1,Klingon,4,4 +RESPONSE_8,4.60517018598809,-0.143853507365118,-0.33166247903554,-0.187082869338697,-0.122474487139159,4.165,19,-6631.669747850211,200,QUESTION_1,Navi,2,2 +RESPONSE_9,5.62941805936734,-0.0955537259311259,-0.15277211058890305,-0.22016804419356895,-0.0947452409476392,4.39856373429084,1,-11808.509487992413,557,QUESTION_1,Esperanto,5,4 +RESPONSE_10,5.1733208763733485,-0.105235660241431,-0.184375525158901,-0.199148328281268,-0.14081913338964902,4.1983002832861205,17,-11592.315357899997,353,QUESTION_1,Esperanto,3,3 +RESPONSE_11,5.2094861528414205,-0.142623900916567,-0.22784329769824505,-0.165294901226822,-0.116881145553046,4.25956284153005,15,-9543.875497870697,366,QUESTION_1,Klingon,4,4 +RESPONSE_12,5.5664342835049805,-0.0625729501677245,-0.131180840833988,0.0,-0.0437269469446626,4.95219885277247,13,-7714.666968654571,523,QUESTION_1,Esperanto,5,5 +RESPONSE_13,6.599870499212838,-0.0879313440357718,-0.0824786098842323,-0.10432810619146,-0.063887656499994,4.40952380952381,18,-8286.345698388934,735,QUESTION_1,Esperanto,4,3 +RESPONSE_14,5.23821344247833,-0.0779205375161998,-0.151686682202236,0.0,-0.103050807985909,4.59115044247788,6,-9360.986354107848,565,QUESTION_1,Navi,4,4 +RESPONSE_15,5.780743515792331,-0.11571934779347802,-0.175682092231577,-0.19245008972987496,0.0,4.41358024691358,16,-11647.41617279202,324,QUESTION_1,Vulcan,3,4 +RESPONSE_16,5.932245187448009,-0.119512183678226,-0.11516335992622,-0.199468790562554,-0.12615514005935402,4.74005305039788,4,-11310.07353041147,377,QUESTION_1,Vulcan,4,4 +RESPONSE_17,5.0689042022202315,-0.14974712591147302,-0.18598708280026705,-0.4415524356847321,-0.112154430818409,4.25471698113208,14,-9880.144982894268,318,QUESTION_1,Navi,4,4 +RESPONSE_18,5.24438902452248,-0.102489200577496,-0.125821836597006,-0.17036357210931902,-0.102733099387503,4.33509234828496,1,-3574.403624590792,379,QUESTION_1,Esperanto,3,3 +RESPONSE_19,4.90897164031976,-0.15701216675512802,-0.33271775878523496,-0.309743376700123,-0.121491347846157,4.12177121771218,3,-8801.02345222067,271,QUESTION_1,Klingon,2,3 +RESPONSE_20,5.613128106388071,-0.10299769004288198,-0.0955200899876088,-0.042717882885838,-0.0739895435500485,4.74270072992701,14,-8849.349326554664,548,QUESTION_1,Navi,4,4 +RESPONSE_21,6.084499413075171,-0.107220707321994,-0.0826662747682189,-0.190909583961339,-0.0826662747682189,4.35535307517084,10,-6208.38187355175,439,QUESTION_1,Vulcan,4,3 +RESPONSE_22,5.70544775397526,-0.10307874659672,-0.12237255246720098,-0.0407908508224002,-0.0706518261083599,4.8752079866888485,13,-4647.680689060316,601,QUESTION_1,Klingon,5,5 +RESPONSE_23,4.9732795075524905,-0.117133148603098,-0.11764705882352902,-0.24956709924231105,-0.0831890330807703,4.41522491349481,18,-8493.435016335656,289,QUESTION_1,Esperanto,3,3 +RESPONSE_24,5.08450514266271,-0.11483183223927,-0.222565953629863,-0.347480983732531,-0.136293255127276,4.6811145510835885,8,-6454.317071248449,323,QUESTION_1,Klingon,3,3 +RESPONSE_25,5.932245187448009,-0.110402014680565,-0.15874173676775302,-0.21850711221876895,-0.08143279274805701,4.13395225464191,14,-4766.856389922178,754,QUESTION_1,Navi,4,4 +RESPONSE_26,5.9571318680274805,-0.091663680930583,-0.0719350001265606,-0.148297801850315,-0.0622975375308383,4.15006468305304,1,-7258.168156975674,773,QUESTION_1,Klingon,4,4 +RESPONSE_27,5.73657229747919,-0.0662986938343081,-0.0695608343640252,-0.1606438657805,-0.0567961834247065,4.20161290322581,16,-8511.089075758706,620,QUESTION_1,Navi,4,4 +RESPONSE_28,5.82008293035236,-0.136357797410945,-0.231111364731404,-0.24361276856704805,-0.0943508195583659,4.281899109792279,11,-10726.061974151535,337,QUESTION_1,Vulcan,4,4 +RESPONSE_29,5.01727983681492,-0.0926146023893406,-0.0813788458771159,-0.293415601548023,-0.0813788458771159,4.35099337748344,13,-8053.846770897318,302,QUESTION_1,Esperanto,3,3 +RESPONSE_30,5.19849703126583,-0.11366621599829302,-0.157676499368291,-0.166205623828633,-0.105117666245527,4.30939226519337,3,-8420.586662169268,362,QUESTION_1,Navi,3,3 +RESPONSE_31,4.7957905455967405,-0.114623600718628,-0.0909090909090909,-0.0524863881081478,-0.104972776216296,4.303030303030299,19,-7281.693412765701,363,QUESTION_1,Vulcan,3,3 +RESPONSE_32,5.30578938138674,-0.105524466349827,-0.211340986102904,-0.22277295967705604,-0.179605302026775,4.16377171215881,19,-8406.190106592592,403,QUESTION_1,Klingon,3,3 +RESPONSE_33,4.26267987704132,-0.0898511226941063,-0.37529331252040105,-0.517306131611464,-0.118678165819385,4.35211267605634,17,-7727.415027567354,71,QUESTION_1,Vulcan,1,1 +RESPONSE_34,5.605802066296,-0.12900316560794,-0.0958706236059213,-0.148522131446501,-0.0606339062590832,4.51102941176471,16,-5082.828837504528,544,QUESTION_1,Vulcan,4,4 +RESPONSE_35,4.945207488773799,-0.0832845773168749,-0.178964995881568,0.0,-0.119309997254379,4.37366548042705,9,-9307.46030612153,281,QUESTION_1,Vulcan,4,3 +RESPONSE_36,4.52720864451838,-0.0734993876109588,-0.254685815986805,-0.24384310418681,-0.12734290799340306,4.75675675675676,15,-11879.050329302772,185,QUESTION_1,Navi,3,3 +RESPONSE_37,5.560681631015531,-0.105236962025889,-0.169841555121689,0.0,-0.131558702896054,4.21153846153846,12,-7221.795746363577,520,QUESTION_1,Klingon,4,4 +RESPONSE_38,6.39859493453521,-0.0880685428443249,-0.168184986499569,-0.0815817016448004,-0.0407908508224002,4.7803660565723805,8,-6916.77047425752,601,QUESTION_1,Vulcan,5,5 +RESPONSE_39,3.78418963391826,-0.209372632515358,-0.238365647311398,-0.533001790889026,-0.21320071635561,3.98863636363636,4,-4706.097844898101,88,QUESTION_1,Vulcan,2,1 +RESPONSE_40,5.11799381241676,-0.0947957644650186,-0.189547207081969,-0.197286988138151,-0.0547175655164583,4.76347305389222,19,-10728.055667962068,334,QUESTION_1,Vulcan,4,3 +RESPONSE_41,5.804007866322004,-0.11227845781988313,-0.19754756910504806,0.001384580160839074,-0.12744134684805405,4.674863805034417,6,-5034.432953817061,680,QUESTION_2,Navi,4,4 +RESPONSE_42,5.8720601652947515,-0.08690438119863073,-0.10401313876856116,-0.0002864254139357557,-0.06821115801063006,4.573443713084811,1,-9749.746133720346,651,QUESTION_2,Esperanto,4,4 +RESPONSE_43,4.660644556928407,-0.11344485321911615,-0.17790145524595927,-0.2714335572828666,-0.09479791862861305,4.685429239359897,10,-7165.506631616553,215,QUESTION_2,Esperanto,4,3 +RESPONSE_44,5.118102377402918,-0.08540410933722904,-0.12242375056583805,-0.23710715317126174,-0.07537424335091739,4.708294147100864,7,-12418.690296394772,342,QUESTION_2,Klingon,3,4 +RESPONSE_45,5.229891027079035,-0.04650315528300876,-0.1521501543593347,-0.3046042304843754,-0.12374981097470127,4.247249758706574,11,-6162.0428654214975,401,QUESTION_2,Vulcan,2,3 +RESPONSE_46,4.267930369673505,-0.14976954923331387,-0.27303065420053635,-0.40952254173983116,-0.0835900460130872,3.909992575498007,13,-5232.625455542854,143,QUESTION_2,Esperanto,2,2 +RESPONSE_47,6.348811607299139,-0.10589115343862064,-0.2830763662215601,0.0004458525062610013,-0.12094001019965979,4.175370463728134,7,-10385.87509687082,543,QUESTION_2,Klingon,4,4 +RESPONSE_48,4.701777482105613,-0.1453316246997682,-0.3322272621370448,-0.18972139400390606,-0.1217732077897799,4.168744400526809,5,-6864.614908204971,200,QUESTION_2,Esperanto,2,2 +RESPONSE_49,5.701639735256522,-0.0957814642530415,-0.15021680331135098,-0.21917728513098209,-0.09369881743517551,4.412759639616131,18,-6029.567409373588,557,QUESTION_2,Navi,5,4 +RESPONSE_50,5.187306324722378,-0.10320231978763127,-0.18537262764431378,-0.19812175941305982,-0.14141856104847747,4.182466417882258,10,-8146.931921957305,353,QUESTION_2,Vulcan,3,3 +RESPONSE_51,5.261744611095715,-0.14294713273454557,-0.2294020382429947,-0.16720254313615204,-0.1159978877767079,4.259553432228078,11,-8986.084993875762,366,QUESTION_2,Klingon,4,4 +RESPONSE_52,5.626477937872781,-0.06315101830389494,-0.1303938055249117,-0.0013801733950718383,-0.04372245117734715,5.00420257106861,10,-6922.795075829423,523,QUESTION_2,Klingon,5,5 +RESPONSE_53,6.622359735153993,-0.08761845002269053,-0.08265775270333232,-0.1048453993756711,-0.06374417391421987,4.402084982004124,15,-5094.2474373975865,735,QUESTION_2,Klingon,4,3 +RESPONSE_54,5.227837853317145,-0.07891017780535535,-0.15118380062592704,-0.00041327907241887126,-0.10422013398167021,4.627782152773747,18,-12490.377127380138,565,QUESTION_2,Vulcan,4,4 +RESPONSE_55,5.924190262669798,-0.11594554719456689,-0.17635559514686078,-0.1957589162000422,-0.0009515209412315562,4.437966350858516,2,-9924.560567511675,324,QUESTION_2,Klingon,3,4 +RESPONSE_56,6.014421903174632,-0.1200660263218193,-0.11606372560168873,-0.19952775533775496,-0.1263155216642745,4.685395818067799,18,-9357.455226438104,377,QUESTION_2,Vulcan,4,4 +RESPONSE_57,5.010378585366719,-0.14826550561432106,-0.18549191561398487,-0.44493490344874365,-0.11212196828097398,4.324560380872834,13,-6909.954620801603,318,QUESTION_2,Esperanto,4,4 +RESPONSE_58,5.177886550421138,-0.10106598697722131,-0.12547025512457216,-0.17019340943589217,-0.10057151549525375,4.272711330289151,2,-4797.839374462829,379,QUESTION_2,Esperanto,3,3 +RESPONSE_59,4.9388114226530115,-0.1553043238493183,-0.33677567006378595,-0.31195192118006876,-0.11958695378558379,4.205350001362887,13,-4675.735681235145,271,QUESTION_2,Vulcan,2,3 +RESPONSE_60,5.552500907115843,-0.10254573487807757,-0.09621160664435797,-0.04366883320406821,-0.07445140820605461,4.677947815369887,1,-8120.792962912871,548,QUESTION_2,Navi,4,4 +RESPONSE_61,6.047305983159986,-0.10825174404629101,-0.08410035137055263,-0.18952412609086533,-0.08299747240513675,4.358495412867743,14,-12006.731456577638,439,QUESTION_2,Esperanto,4,3 +RESPONSE_62,5.71126931399781,-0.10179478748602382,-0.12204011423131007,-0.040416808394696635,-0.07145518084908045,4.905100406285923,13,-9964.15064766773,601,QUESTION_2,Vulcan,5,5 +RESPONSE_63,4.901834670344162,-0.11654405610912195,-0.11753045645064773,-0.2509743837582159,-0.0839403623508296,4.480974960551953,3,-10540.410143205598,289,QUESTION_2,Klingon,3,3 +RESPONSE_64,5.095219062033827,-0.11555687902521843,-0.22337406814265887,-0.35277026823075563,-0.1377887711163444,4.6914736972585835,18,-5665.9409641272305,323,QUESTION_2,Klingon,3,3 +RESPONSE_65,5.879038632945785,-0.11117952312897957,-0.15967549809237036,-0.21853831238745886,-0.08149086182894617,4.0706659058111265,9,-12184.112015521276,754,QUESTION_2,Klingon,4,4 +RESPONSE_66,5.854407498388798,-0.09188411607577093,-0.07191633895490418,-0.14866368338353475,-0.06311945329350363,4.183343562628803,18,-8217.195580128606,773,QUESTION_2,Navi,4,4 +RESPONSE_67,5.8393911313109,-0.06591044914005889,-0.06995893265428846,-0.160909457442852,-0.057114294841595685,4.1837784550482455,10,-7010.888032984205,620,QUESTION_2,Vulcan,4,4 +RESPONSE_68,5.743681962191419,-0.1358767263379277,-0.228556227613391,-0.24332259169628975,-0.09419518358190605,4.329807350884032,18,-4563.021382512034,337,QUESTION_2,Klingon,4,4 +RESPONSE_69,5.045169762846371,-0.09429607392733567,-0.08236191502457627,-0.29950391802352294,-0.0818928231020527,4.419284309829859,2,-8337.573446366074,302,QUESTION_2,Navi,3,3 +RESPONSE_70,5.163049451843117,-0.11470071933459283,-0.15978889307012403,-0.16787225939255013,-0.10602910976252362,4.370918827703467,8,-9313.097349483209,362,QUESTION_2,Esperanto,3,3 +RESPONSE_71,4.871215993504256,-0.11548159041113068,-0.08902255872115292,-0.05221159771318208,-0.10602519408504922,4.2439997036416415,2,-12074.286648690933,363,QUESTION_2,Vulcan,3,3 +RESPONSE_72,5.300236789241013,-0.10563856460148643,-0.21448871925280927,-0.22094284125353983,-0.17743684063982076,4.152678679308845,18,-3335.9869841940854,403,QUESTION_2,Navi,3,3 +RESPONSE_73,4.34155641177576,-0.08960423082408912,-0.3796127624059438,-0.5180755823183352,-0.11664641103330715,4.386024971922706,7,-11531.040413446352,71,QUESTION_2,Klingon,1,1 +RESPONSE_74,5.587115101185276,-0.12869697689815987,-0.09546886833872449,-0.148589516549389,-0.0610126192584671,4.506391428691971,5,-9377.278523107123,544,QUESTION_2,Vulcan,4,4 +RESPONSE_75,4.954774696552758,-0.08350098703941,-0.17822514514564072,-0.0007044929255297609,-0.11956310840728994,4.348015479710287,6,-5381.20048067755,281,QUESTION_2,Navi,4,3 +RESPONSE_76,4.504719964750152,-0.07372903069780709,-0.25285824932070944,-0.2396238491118196,-0.1288628273318789,4.768184991137497,17,-6413.573845633859,185,QUESTION_2,Klingon,3,3 +RESPONSE_77,5.618108081172188,-0.1047649639984788,-0.1693970375838369,0.00022892725550405343,-0.1306935914255415,4.169504963601735,18,-6806.096875364829,520,QUESTION_2,Navi,4,4 +RESPONSE_78,6.28727193687608,-0.08812097177816666,-0.16859329132909398,-0.08069022489569115,-0.040709236536390525,4.78950467819536,5,-6086.896496713913,601,QUESTION_2,Klingon,5,5 +RESPONSE_79,3.755586375821629,-0.2103567149033811,-0.23956049834079726,-0.534886333458205,-0.210874142518448,3.950632515843758,6,-8078.370253056693,88,QUESTION_2,Esperanto,2,1 +RESPONSE_80,5.072898568459361,-0.09446438997153847,-0.19087049106343434,-0.1973474740893821,-0.05498514648158415,4.833543579580617,8,-7413.623263267944,334,QUESTION_2,Vulcan,4,3 +RESPONSE_81,5.870850194529269,-0.11252862053654297,-0.19488050672771395,0.001454343476390633,-0.12948938150910497,4.633829846053602,8,-11250.066152780984,680,QUESTION_3,Navi,4,4 +RESPONSE_82,5.775726386388563,-0.08680979231133269,-0.10259900220128933,0.0007043082981680985,-0.06849762879918728,4.534367026810253,20,-8954.03932877404,651,QUESTION_3,Vulcan,4,4 +RESPONSE_83,4.71325547098978,-0.11283875348536636,-0.17827586097446654,-0.26879756243615305,-0.09543195307514239,4.638034439887701,4,-7473.0197887933255,215,QUESTION_3,Vulcan,4,3 +RESPONSE_84,5.080506375036875,-0.08603414211770848,-0.12404997510147939,-0.24097026838686209,-0.07722200658522545,4.685028702151052,2,-5695.647710971749,342,QUESTION_3,Vulcan,3,4 +RESPONSE_85,5.272462456711578,-0.04603709247572576,-0.15088363125288853,-0.30705603678204557,-0.1218910116665069,4.336671056337276,5,-8108.209641355039,401,QUESTION_3,Navi,2,3 +RESPONSE_86,4.268775286992498,-0.15020464640555214,-0.27511924357696427,-0.40818527198456933,-0.08306727900793953,3.7828534797950013,6,-7932.7581306286265,143,QUESTION_3,Esperanto,2,2 +RESPONSE_87,6.227514197223741,-0.10691307661818725,-0.2780532249787314,-0.001813277132405328,-0.12008612620184456,4.266625054808037,5,-4378.666713102711,543,QUESTION_3,Esperanto,4,4 +RESPONSE_88,4.556542060026307,-0.14270574211090206,-0.3320849022259334,-0.1874531264667034,-0.12275426003525998,4.12519578157565,16,-9001.053165380416,200,QUESTION_3,Vulcan,2,2 +RESPONSE_89,5.686914439284896,-0.09582996772774224,-0.15441437299106264,-0.21921499267044317,-0.09439990979434612,4.392683988953526,9,-3806.3139990711684,557,QUESTION_3,Esperanto,5,4 +RESPONSE_90,5.261766872049131,-0.10452323838105551,-0.1817052713536288,-0.20022485601074833,-0.14295232732452187,4.153067656452709,10,-6812.741366180402,353,QUESTION_3,Vulcan,3,3 +RESPONSE_91,5.21102617863412,-0.14311336337308156,-0.22847750926843335,-0.16480623882767512,-0.11696911180772684,4.312803224430802,9,-5691.292243217336,366,QUESTION_3,Vulcan,4,4 +RESPONSE_92,5.573327867965382,-0.06226808841298423,-0.13254094827526947,0.0020798442368337015,-0.04385684137446748,4.980483274789093,19,-10463.76066786842,523,QUESTION_3,Klingon,5,5 +RESPONSE_93,6.618972508141033,-0.08725808419552293,-0.08231245551516882,-0.10625028156470338,-0.06435238450155287,4.4226222307333245,2,-7711.792788854956,735,QUESTION_3,Vulcan,4,3 +RESPONSE_94,5.222420635227893,-0.07963522915292195,-0.15092558547798965,-0.0005111660166205719,-0.10330418451081444,4.631179086269016,9,-11687.759477426489,565,QUESTION_3,Vulcan,4,4 +RESPONSE_95,5.804700940834062,-0.11872349426734262,-0.17284775168588457,-0.193568099905743,-0.00012794894211963254,4.406354721879594,4,-9697.206628024762,324,QUESTION_3,Navi,3,4 +RESPONSE_96,5.9885930870758495,-0.11894369429198509,-0.11734882724739025,-0.1973489629856295,-0.12662104438915486,4.772736815815229,9,-12143.252230596087,377,QUESTION_3,Vulcan,4,4 +RESPONSE_97,5.095356204900446,-0.15213542132678293,-0.18357095701813392,-0.4414727129082793,-0.11509624634400727,4.339155664748932,12,-7152.312445674943,318,QUESTION_3,Vulcan,4,4 +RESPONSE_98,5.141113218491709,-0.10362690934557732,-0.12512580979247903,-0.17032473194122444,-0.10055942035526572,4.329964719334843,8,-5462.037555437275,379,QUESTION_3,Klingon,3,3 +RESPONSE_99,4.933654441987073,-0.15857978642676546,-0.33338439972595896,-0.31494641956701963,-0.1206584498356429,4.161139487802116,17,-11295.773894040254,271,QUESTION_3,Esperanto,2,3 +RESPONSE_100,5.593233920830482,-0.10534171827334843,-0.09499886551726054,-0.042370832979868836,-0.07410603563568373,4.686710639576134,9,-9070.158476934983,548,QUESTION_3,Vulcan,4,4 +RESPONSE_101,6.057302279832855,-0.10555577645037259,-0.08244945219109827,-0.19210446548691446,-0.08152441562706947,4.419397407105573,3,-3868.7695408581485,439,QUESTION_3,Navi,4,3 +RESPONSE_102,5.641719135742499,-0.10426342936212103,-0.12210447065991985,-0.04130084692683778,-0.07060348149506565,4.8741569950246975,15,-5564.043152374374,601,QUESTION_3,Navi,5,5 +RESPONSE_103,4.934076817082659,-0.11691231674174037,-0.11655017302086022,-0.2480319235224862,-0.08292903716102688,4.446611046265905,10,-6626.41629755686,289,QUESTION_3,Esperanto,3,3 +RESPONSE_104,5.138184915733327,-0.11369618700696842,-0.2186865123714815,-0.34790280692108,-0.13567413867471523,4.643111622330122,13,-12563.160001760189,323,QUESTION_3,Navi,3,3 +RESPONSE_105,5.854856604246296,-0.11215314111498026,-0.15893969355483298,-0.22130595948671541,-0.08207486741614828,4.0408881813725674,14,-7278.237131591467,754,QUESTION_3,Esperanto,4,4 +RESPONSE_106,5.989284607854202,-0.09209668262911852,-0.07124871744559949,-0.14781641264353726,-0.06233855443781622,4.1033307969383985,5,-10971.474843459346,773,QUESTION_3,Klingon,4,4 +RESPONSE_107,5.756477701422786,-0.06642389649630351,-0.06906552578264741,-0.15956663718867137,-0.057601825839945456,4.188677534304624,10,-3994.8465053485843,620,QUESTION_3,Klingon,4,4 +RESPONSE_108,5.870961681715444,-0.13606990917447115,-0.23245027758628448,-0.24324939253810723,-0.09448267798021509,4.240712184423488,1,-7692.412742533708,337,QUESTION_3,Klingon,4,4 +RESPONSE_109,4.938248680260479,-0.09112191219656221,-0.0819988057576822,-0.29245594532041413,-0.08183078724962463,4.380180970547859,8,-6024.372806312614,302,QUESTION_3,Navi,3,3 +RESPONSE_110,5.190180285649064,-0.11302203495017352,-0.1550567009747582,-0.1667922660076093,-0.10564527050112762,4.3121237056763215,1,-8721.816076866622,362,QUESTION_3,Vulcan,3,3 +RESPONSE_111,4.764937631079964,-0.11535854117353883,-0.08859300754322025,-0.05296166685993762,-0.10424519046180415,4.2747681772648685,6,-4906.353651757593,363,QUESTION_3,Klingon,3,3 +RESPONSE_112,5.267833308846324,-0.1039006833600877,-0.2114232237922227,-0.2190388717948239,-0.18004598248819711,4.1517649275395625,16,-10849.37198894878,403,QUESTION_3,Vulcan,3,3 +RESPONSE_113,4.33512237089551,-0.08957288612620037,-0.3766308924403732,-0.5137832975074641,-0.12018470426607501,4.342547015466895,15,-6797.613304021446,71,QUESTION_3,Navi,1,1 +RESPONSE_114,5.677715381944702,-0.12958219461438358,-0.09765627978905708,-0.1495673078411187,-0.060792963459530724,4.520062656173169,19,-9253.508720036241,544,QUESTION_3,Navi,4,4 +RESPONSE_115,4.922827573168012,-0.08181118938457967,-0.1742188457690199,0.0001020797487098907,-0.11652100856945367,4.336686128248164,13,-12904.300220886138,281,QUESTION_3,Klingon,4,3 +RESPONSE_116,4.545036797915292,-0.07330226189473582,-0.2550327754171689,-0.2409290421990283,-0.12833836971083123,4.882474160137356,13,-10339.6338220015,185,QUESTION_3,Klingon,3,3 +RESPONSE_117,5.584355710944149,-0.1067757632590403,-0.1696653456292538,-0.0002779593857816089,-0.13145332470980084,4.19748267215859,16,-5511.3443723061155,520,QUESTION_3,Vulcan,4,4 +RESPONSE_118,6.390273726126506,-0.08643856129769649,-0.16799084682638513,-0.08162425897738,-0.04133720801457021,4.773410224313219,17,-6332.601680017214,601,QUESTION_3,Navi,5,5 +RESPONSE_119,3.769290024258319,-0.20581168662831628,-0.23774906578439398,-0.5311166509602837,-0.21231923593895558,4.0242695927723,4,-5243.913779081079,88,QUESTION_3,Klingon,2,1 +RESPONSE_120,5.085184412844011,-0.0944607647119814,-0.18672723114768786,-0.19809253214296282,-0.05460569437727462,4.8248530116431265,10,-12561.993958888681,334,QUESTION_3,Navi,4,3 +RESPONSE_121,5.84825097987591,-0.11299602669563746,-0.19541051469870252,-0.00023102356637828622,-0.12660279242264538,4.531326605209457,13,-12396.341281345602,680,QUESTION_4,Esperanto,4,4 +RESPONSE_122,5.781002155563445,-0.08513360232400781,-0.10413221436792436,-0.00033892928224183783,-0.06743893980819063,4.535529206184901,17,-12762.756561359316,651,QUESTION_4,Vulcan,4,4 +RESPONSE_123,4.674630671577134,-0.11386376463677876,-0.1810544292309035,-0.2719285585122412,-0.09572375011893737,4.666653019534669,4,-6943.068522193119,215,QUESTION_4,Klingon,4,3 +RESPONSE_124,5.2370957559928115,-0.08668241798284443,-0.11932628947679563,-0.23844702197508352,-0.07790360588058472,4.681865648481826,20,-7907.632934015247,342,QUESTION_4,Vulcan,3,4 +RESPONSE_125,5.260896927162302,-0.046277092791338965,-0.15254383796043905,-0.306431173571708,-0.12300213160217482,4.221733834654363,13,-8428.15980376179,401,QUESTION_4,Esperanto,2,3 +RESPONSE_126,4.2551708344154004,-0.14943952159365453,-0.27675275276898526,-0.4092151466191439,-0.08339660605894848,3.9159447052406806,10,-4570.2661270429235,143,QUESTION_4,Vulcan,2,2 +RESPONSE_127,6.262412496990317,-0.1054393827805709,-0.28329405375457944,0.00043156812647400777,-0.12024872092103646,4.1903591442523895,8,-9831.67925901914,543,QUESTION_4,Vulcan,4,4 +RESPONSE_128,4.577353239127284,-0.14300931945340317,-0.3308010936238479,-0.18406375387394036,-0.12207334359632843,4.1755357213535405,3,-5784.8138179102125,200,QUESTION_4,Navi,2,2 +RESPONSE_129,5.748907521879575,-0.09572132023025089,-0.15205477001922973,-0.22082278998728982,-0.09404335512738043,4.377098236419134,14,-7232.756749069753,557,QUESTION_4,Navi,5,4 +RESPONSE_130,5.174371346116375,-0.10569177922447856,-0.1834679677776154,-0.1958843622307346,-0.14194836912851252,4.225058995333766,10,-5978.798341794686,353,QUESTION_4,Navi,3,3 +RESPONSE_131,5.16435845525739,-0.1425616141902891,-0.22667635147813178,-0.16507324639725418,-0.11873044838664415,4.179078215731841,4,-10250.584734774342,366,QUESTION_4,Vulcan,4,4 +RESPONSE_132,5.546623990357694,-0.06405523636114048,-0.13267895260875107,0.00017203736635766856,-0.0444354066603075,4.8669293476036115,5,-4782.459058444689,523,QUESTION_4,Esperanto,5,5 +RESPONSE_133,6.536885142582891,-0.08933184513692874,-0.08257337274630307,-0.10575220273771338,-0.0638579091992722,4.38790638168771,9,-8658.597351315886,735,QUESTION_4,Navi,4,3 +RESPONSE_134,5.3215777296351865,-0.0798231004813728,-0.14976636685165307,-0.0004212656827400207,-0.10478188519778152,4.631379510446408,11,-12064.293330151793,565,QUESTION_4,Navi,4,4 +RESPONSE_135,5.780777468297798,-0.11602338166598479,-0.17731458298145733,-0.1907552969736076,0.0013958923270092252,4.391573845982787,20,-5505.705263185724,324,QUESTION_4,Navi,3,4 +RESPONSE_136,5.904717687369006,-0.12049531279425446,-0.11494349542850654,-0.20240610125392314,-0.1272211149399899,4.772456186843005,19,-10299.513209452729,377,QUESTION_4,Esperanto,4,4 +RESPONSE_137,4.991668762143316,-0.14981042153329546,-0.18412431870281182,-0.4382067499201125,-0.1112414859724597,4.270362613436694,13,-10558.494812215939,318,QUESTION_4,Esperanto,4,4 +RESPONSE_138,5.252511028965032,-0.10289021027843226,-0.12594441410546456,-0.17144811514927663,-0.10278420120002821,4.381577489974512,12,-9244.440511022112,379,QUESTION_4,Vulcan,3,3 +RESPONSE_139,4.8677044024292035,-0.15665012731609945,-0.33977003436008757,-0.30979243435380455,-0.12084228426359335,4.123278963357781,15,-11869.537795045138,271,QUESTION_4,Vulcan,2,3 +RESPONSE_140,5.613727913049584,-0.10265927682247981,-0.09761859161060316,-0.04288354913268517,-0.0745866459214415,4.750299687283833,9,-6450.1777653730305,548,QUESTION_4,Esperanto,4,4 +RESPONSE_141,6.051415532209252,-0.10632214570257734,-0.08173853773042804,-0.1892111312362371,-0.08197052171795015,4.412211657034914,10,-7048.948519919755,439,QUESTION_4,Klingon,4,3 +RESPONSE_142,5.743985505709358,-0.10401457958373628,-0.11991061003676412,-0.040328218465034545,-0.07104618525614442,4.937713188669968,3,-9285.920857631052,601,QUESTION_4,Navi,5,5 +RESPONSE_143,4.924872478723512,-0.11718204643718069,-0.11729175206231218,-0.2452455199949028,-0.08371242308199618,4.356748583357014,8,-12188.868166615195,289,QUESTION_4,Vulcan,3,3 +RESPONSE_144,5.126114094011395,-0.11712196439959295,-0.22365577372902704,-0.34993253103000443,-0.1346881742661583,4.69143746182528,10,-11428.178523407785,323,QUESTION_4,Navi,3,3 +RESPONSE_145,5.925362757615068,-0.11028259916297627,-0.1595569535199399,-0.22131669946580412,-0.08212096559056403,4.140754969185974,10,-10236.690067540858,754,QUESTION_4,Klingon,4,4 +RESPONSE_146,5.915976533177087,-0.09220384188432325,-0.07270752524237589,-0.14924425333685418,-0.06247286704016539,4.218933630100374,8,-10220.648972185949,773,QUESTION_4,Esperanto,4,4 +RESPONSE_147,5.7961727294815635,-0.06695172286372725,-0.06802805125333398,-0.15951809799384395,-0.055819313462078836,4.1607375480510855,9,-5133.191838634405,620,QUESTION_4,Navi,4,4 +RESPONSE_148,5.8771937656798245,-0.13777070609046138,-0.23180199125731832,-0.2414661977439231,-0.09536744921463894,4.249831766167871,8,-7012.674330949985,337,QUESTION_4,Navi,4,4 +RESPONSE_149,5.048217428061751,-0.09072555527603497,-0.08028340659530937,-0.2953322034364369,-0.08250047560611096,4.384270172531184,17,-6060.621971529094,302,QUESTION_4,Vulcan,3,3 +RESPONSE_150,5.1575595412682835,-0.11441500024822823,-0.15668815077641993,-0.16839948139124988,-0.10542791516539347,4.309677565720284,14,-7575.220061326975,362,QUESTION_4,Navi,3,3 +RESPONSE_151,4.796790791006131,-0.11532049336821862,-0.09096891886785116,-0.05291610078320796,-0.10326292104351485,4.311279225075418,19,-11990.70508814871,363,QUESTION_4,Esperanto,3,3 +RESPONSE_152,5.337589094426573,-0.10461014654898163,-0.21523782901737798,-0.22241050314571492,-0.18028319986871216,4.1649798951937695,17,-11787.272940529656,403,QUESTION_4,Klingon,3,3 +RESPONSE_153,4.24655862368342,-0.09051734135479156,-0.3748805929220525,-0.5138414221097989,-0.11864641000458014,4.2698915724114075,19,-5337.275427902927,71,QUESTION_4,Esperanto,1,1 +RESPONSE_154,5.602692597016124,-0.13090023646058396,-0.09394692094045382,-0.1493747679206672,-0.06119309533745335,4.439476602259205,10,-13900.906281286168,544,QUESTION_4,Klingon,4,4 +RESPONSE_155,5.0432950885835615,-0.08269452763975522,-0.1803212831757586,0.0007436871197925536,-0.12100033052515591,4.406828783843905,11,-7102.8935179186665,281,QUESTION_4,Vulcan,4,3 +RESPONSE_156,4.530734798056303,-0.07359384343606747,-0.2520982135972788,-0.24233929299109352,-0.12624326631735067,4.768078918976055,12,-11105.77004985703,185,QUESTION_4,Esperanto,3,3 +RESPONSE_157,5.583179282231734,-0.10540601450228101,-0.17254627025794045,0.0009718668340412516,-0.13182761648635224,4.2152375778688205,6,-5528.750941475834,520,QUESTION_4,Navi,4,4 +RESPONSE_158,6.34427025504059,-0.08711989473701909,-0.1676932698768474,-0.08216248464827669,-0.0416514725198558,4.74449766979887,11,-8691.165306069848,601,QUESTION_4,Navi,5,5 +RESPONSE_159,3.781234567836048,-0.2106569188040306,-0.2351786219863613,-0.5369230466093129,-0.21432233117128927,4.00778594097705,13,-6545.655780453577,88,QUESTION_4,Klingon,2,1 +RESPONSE_160,5.091006004569724,-0.09547048624050507,-0.19000828810063863,-0.19918777681698094,-0.0546889066820984,4.727331968986029,12,-9024.241500295573,334,QUESTION_4,Vulcan,4,3 +RESPONSE_161,5.808565232710129,-0.11304446824578664,-0.19464223801612127,-0.0009961724067813502,-0.126216420712024,4.690506610716851,18,-9851.950130789224,680,QUESTION_5,Vulcan,4,4 +RESPONSE_162,5.78693231254012,-0.08587134553309879,-0.10310622041424201,-0.0014390314204515066,-0.06753473683028184,4.5053303673042,16,-11917.666935782427,651,QUESTION_5,Esperanto,4,4 +RESPONSE_163,4.649081400043202,-0.11428349730478804,-0.18244278977200615,-0.2746263484186033,-0.09396221677346658,4.614375030754431,19,-5608.074011635437,215,QUESTION_5,Navi,4,3 +RESPONSE_164,5.144104789978727,-0.08671391666744432,-0.12143776513343785,-0.24048973132817109,-0.07728522789373887,4.658488272937101,17,-9630.909990214588,342,QUESTION_5,Navi,3,4 +RESPONSE_165,5.312822572812442,-0.0463605367591214,-0.14856142343011808,-0.30375046964895036,-0.12209696134357641,4.279402693312399,3,-8293.271034021986,401,QUESTION_5,Klingon,2,3 +RESPONSE_166,4.259940105212093,-0.1509063669940506,-0.2826433887592033,-0.4143244288840974,-0.08387526351593641,3.8464665182983016,16,-11862.860825771686,143,QUESTION_5,Vulcan,2,2 +RESPONSE_167,6.256850293958461,-0.10686017858308594,-0.28014740134343546,6.0521341615795214e-05,-0.122414997362171,4.299990355620265,12,-12332.96499905688,543,QUESTION_5,Klingon,4,4 +RESPONSE_168,4.649831041752774,-0.13995455647701038,-0.33350884731514147,-0.19104985238867367,-0.12110223632669476,4.18748206509707,9,-8212.823655139078,200,QUESTION_5,Vulcan,2,2 +RESPONSE_169,5.5681451047062005,-0.09552415111699258,-0.15232048910480303,-0.22499477408389867,-0.09463125125845938,4.377692973653219,16,-8011.3293553588,557,QUESTION_5,Esperanto,5,4 +RESPONSE_170,5.060598375778115,-0.10618881198141344,-0.18804375592127262,-0.1993705802804871,-0.13912565943973432,4.214332116106197,18,-7851.892258926941,353,QUESTION_5,Navi,3,3 +RESPONSE_171,5.212287641963099,-0.14275515727180504,-0.22583933955634133,-0.1660861241971709,-0.11736025177538724,4.25057502133225,6,-9403.339129222737,366,QUESTION_5,Navi,4,4 +RESPONSE_172,5.554225003554914,-0.06345610828012938,-0.13078851994902388,-0.0004189253622278274,-0.04353708522957888,4.934478051334714,8,-6088.041676099802,523,QUESTION_5,Navi,5,5 +RESPONSE_173,6.559851040785911,-0.08817692509067042,-0.08258503357349224,-0.10618804231020895,-0.06350601021970587,4.451859904312221,5,-8544.917479459045,735,QUESTION_5,Navi,4,3 +RESPONSE_174,5.274603591094711,-0.07725131664577524,-0.1516288073936409,0.00202099422535066,-0.10471137042457289,4.679753926088708,16,-11004.587898312104,565,QUESTION_5,Esperanto,4,4 +RESPONSE_175,5.770949397655356,-0.11457439847667931,-0.17447788270786735,-0.19119458751653035,-0.0004057499048472584,4.4084038646318575,16,-5024.3120028157,324,QUESTION_5,Esperanto,3,4 +RESPONSE_176,5.782108486883898,-0.11977456718713127,-0.11476278906919107,-0.20118001332655194,-0.12566933382049322,4.690570350211557,12,-6222.754641274607,377,QUESTION_5,Klingon,4,4 +RESPONSE_177,4.962609976719809,-0.14931437260938166,-0.18753844439688372,-0.4332751571951332,-0.11441956018800975,4.3527824531888974,14,-13125.978953559357,318,QUESTION_5,Vulcan,4,4 +RESPONSE_178,5.2456314253881375,-0.10088387798317802,-0.12470821942154614,-0.1676497719132685,-0.10367033024397893,4.367178133681857,12,-11458.40072344777,379,QUESTION_5,Klingon,3,3 +RESPONSE_179,4.922848507212938,-0.15317391045454093,-0.33627150945873535,-0.3026317808814517,-0.12309408048655708,4.110182000509459,19,-10133.776336595816,271,QUESTION_5,Vulcan,2,3 +RESPONSE_180,5.568663568246954,-0.10432318603431262,-0.09549952672872096,-0.04323482564617868,-0.07395212648147716,4.794746063908469,2,-11260.878159207874,548,QUESTION_5,Klingon,4,4 +RESPONSE_181,6.062293729549855,-0.10595710585898653,-0.08136704116987735,-0.18812610188066162,-0.08057851385005194,4.393466256499823,19,-11261.802438020059,439,QUESTION_5,Vulcan,4,3 +RESPONSE_182,5.6816398005526105,-0.10345013719991929,-0.12129083346844902,-0.040856401174379414,-0.07024340724451147,4.835256146956064,10,-6988.860776088699,601,QUESTION_5,Vulcan,5,5 +RESPONSE_183,4.957106082210847,-0.11493992465722333,-0.11784481795029911,-0.24906523695192612,-0.08412624750892622,4.443031923956395,14,-8152.89400252864,289,QUESTION_5,Navi,3,3 +RESPONSE_184,5.100539583345083,-0.11532730144034715,-0.22421276627421577,-0.34613909295533885,-0.13653180516540475,4.710984676725214,18,-5231.525598747583,323,QUESTION_5,Klingon,3,3 +RESPONSE_185,5.933549963814808,-0.1088543526505199,-0.15652925289523043,-0.21767278759883646,-0.08150566376803878,4.128676142361032,14,-6569.936537515637,754,QUESTION_5,Esperanto,4,4 +RESPONSE_186,6.023146424549076,-0.09052627675155289,-0.07189096402255658,-0.14926458357089697,-0.061984825498238326,4.173831731024361,9,-8598.629983759458,773,QUESTION_5,Esperanto,4,4 +RESPONSE_187,5.756246983938716,-0.0655898871060833,-0.06960524414553754,-0.16303370129076336,-0.05659297408720125,4.215949039990627,6,-6539.265608337719,620,QUESTION_5,Navi,4,4 +RESPONSE_188,5.805889146201376,-0.13784420064412284,-0.23326555806100832,-0.23991784905949404,-0.09517797164434391,4.37206664442475,7,-5044.157533355388,337,QUESTION_5,Vulcan,4,4 +RESPONSE_189,5.109673570342725,-0.0925669263733389,-0.0803273349792988,-0.29408787128437264,-0.0826158549519689,4.273942855860416,19,-4980.733597374792,302,QUESTION_5,Klingon,3,3 +RESPONSE_190,5.174673294882774,-0.11467851556173536,-0.15386365995322968,-0.16593317889630527,-0.10454497375378811,4.241833201432851,6,-6040.119559988035,362,QUESTION_5,Klingon,3,3 +RESPONSE_191,4.8711514056391785,-0.11347783247316807,-0.0905955837220561,-0.05170091389935735,-0.10403583239926009,4.347811894331173,7,-11006.788049494837,363,QUESTION_5,Vulcan,3,3 +RESPONSE_192,5.231194913227801,-0.10470272752712328,-0.21399262614850614,-0.22410792771246485,-0.180721536622879,4.1475509814112,9,-4338.532726237085,403,QUESTION_5,Esperanto,3,3 +RESPONSE_193,4.262989545520528,-0.09096530092660604,-0.36632787828213154,-0.5212385340698211,-0.11877163586401654,4.4599876536575636,9,-6493.360571703143,71,QUESTION_5,Klingon,1,1 +RESPONSE_194,5.7083889289260705,-0.13000488683280884,-0.09522960931141106,-0.14862647874459559,-0.06072945372868957,4.472606925832167,5,-8798.69066643964,544,QUESTION_5,Navi,4,4 +RESPONSE_195,4.977245220892741,-0.0840397498757009,-0.17854836935355986,-0.00014426274554185444,-0.11908346023412753,4.381323390838402,11,-10245.038973516486,281,QUESTION_5,Vulcan,4,3 +RESPONSE_196,4.550143162287714,-0.07318163017026906,-0.25586629579956816,-0.243372248303419,-0.12629983976217,4.69771884266387,5,-7610.593936222832,185,QUESTION_5,Vulcan,3,3 +RESPONSE_197,5.651357379300722,-0.10567642881701801,-0.17015740329788193,-3.66583331129061e-06,-0.13244495456538702,4.237412366823095,1,-6182.833971631509,520,QUESTION_5,Esperanto,4,4 +RESPONSE_198,6.541219952879253,-0.08809093313843414,-0.16858161822993545,-0.08095784189805204,-0.040308777007569706,4.929782858285784,9,-4285.628480025826,601,QUESTION_5,Esperanto,5,5 +RESPONSE_199,3.770318292042199,-0.2109171766291959,-0.2389273085943553,-0.5384839868860176,-0.21642264568297784,4.014736799431227,14,-9203.661060223181,88,QUESTION_5,Vulcan,2,1 +RESPONSE_200,5.1214694720159315,-0.0941787346066672,-0.18672004740197287,-0.19812466226550068,-0.055504320978549966,4.863284225856278,13,-9905.56403465048,334,QUESTION_5,Vulcan,4,3 diff --git a/example/train.csv b/example/train.csv new file mode 100644 index 000000000..f0fb1575a --- /dev/null +++ b/example/train.csv @@ -0,0 +1,501 @@ +ID,FEATURE1,FEATURE2,FEATURE3,FEATURE4,FEATURE5,FEATURE6,FEATURE7,FEATURE8,LENGTH,QUESTION,L1,score,score2 +RESPONSE_1,4.93806460126142,-0.0584019308051746,-0.0846667513334603,-0.316793975540994,-0.05986843400892501,4.65591397849462,16,-7940.981205828437,279,QUESTION_1,Esperanto,3,3 +RESPONSE_2,5.37989735354046,-0.0781006575640341,-0.15920308451727802,-0.271537693320852,-0.107334697685273,4.76267281105991,10,-6683.432480150636,434,QUESTION_1,Klingon,4,4 +RESPONSE_3,4.87137322676275,-0.0600602560901643,-0.13840913308956698,-0.3032392174315611,0.0,4.681992337164751,12,-11848.82863236471,261,QUESTION_1,Navi,3,3 +RESPONSE_4,4.71849887129509,-0.053042883416763005,-0.211288563682129,-0.30618621784789696,-0.0668153104781061,4.42410714285714,18,-9644.787763962237,224,QUESTION_1,Esperanto,3,3 +RESPONSE_5,4.62497281328427,-0.116778484607045,-0.121267812518166,-0.261968415997792,-0.121267812518166,5.0,2,-10536.346496314658,204,QUESTION_1,Navi,3,3 +RESPONSE_6,5.52942908751142,-0.0614571044428799,-0.204124145231932,0.0,-0.0996023841111995,4.57539682539683,3,-12407.017493581072,504,QUESTION_1,Esperanto,4,4 +RESPONSE_7,5.0562458053483095,-0.0722739316492093,-0.11286652959662,-0.178457652562062,-0.05643326479831,4.5796178343949006,11,-12841.050973951487,314,QUESTION_1,Navi,3,3 +RESPONSE_8,4.774912960575191,-0.107799232277336,-0.20541200750444,-0.0649569802461631,-0.145248223443532,4.953586497890299,7,-9122.018595513759,237,QUESTION_1,Klingon,3,4 +RESPONSE_9,5.46806014113513,-0.118139961125272,-0.259827920984652,-0.225017580185205,-0.11250879009260198,3.85021097046413,7,-11197.50021337257,474,QUESTION_1,Vulcan,3,3 +RESPONSE_10,6.26909628370626,-0.11236058018844802,-0.32856437033638497,0.0,-0.0615457454896664,4.276515151515151,10,-10903.922373691099,528,QUESTION_1,Vulcan,2,2 +RESPONSE_11,6.15060276844628,-0.081019800496933,-0.146020415081142,0.0,-0.122169444356305,4.19189765458422,20,-4307.599071824252,469,QUESTION_1,Navi,3,3 +RESPONSE_12,5.29078910012725,-0.0780097820250321,-0.18778840120519694,0.0,-0.0501885613228496,4.56926952141058,13,-9509.005831482078,397,QUESTION_1,Navi,4,4 +RESPONSE_13,5.1817835502920895,-0.0700284810505032,-0.198307444884526,-0.158999682000954,-0.0749531688995861,4.47191011235955,17,-5535.571052102188,356,QUESTION_1,Klingon,3,3 +RESPONSE_14,5.48893772615669,-0.114937232629007,-0.0642824346533225,0.0,-0.0909090909090909,4.5495867768595,20,-7160.043414710473,484,QUESTION_1,Navi,4,4 +RESPONSE_15,4.44265125649032,-0.157277464536837,-0.391076944437521,-0.351467511677404,-0.076696498884737,4.7764705882352905,15,-6300.537735570257,170,QUESTION_1,Esperanto,2,2 +RESPONSE_16,5.31073988654659,-0.11386768810566,-0.149071198499986,-0.131468439624436,-0.0702728368926307,4.02962962962963,20,-4841.610815387637,405,QUESTION_1,Klingon,3,4 +RESPONSE_17,6.200509174042691,-0.0763170500114337,-0.2776312437480169,0.0,-0.162385862552742,3.68965517241379,2,-4463.336392987499,493,QUESTION_1,Vulcan,3,3 +RESPONSE_18,5.69204721843778,-0.0675502901300752,-0.0711268017165705,-0.16931565460334902,0.0,4.82462057335582,6,-7712.672518603857,593,QUESTION_1,Vulcan,4,5 +RESPONSE_19,4.52720864451838,-0.05803423019770401,-0.194519505031855,-0.311925146946022,0.0,4.19459459459459,7,-9065.008526892165,185,QUESTION_1,Navi,3,3 +RESPONSE_20,4.753590191106361,-0.15527336698744598,-0.30794088102572,-0.185695338177052,-0.13130643285972302,4.44827586206897,7,-13950.786157620996,232,QUESTION_1,Navi,3,3 +RESPONSE_21,4.5849674786705705,-0.0962131452737853,-0.188982236504614,0.0,0.0,4.581632653061219,2,-4785.19916161297,196,QUESTION_1,Vulcan,3,3 +RESPONSE_22,4.7791234931115305,-0.12348268441809103,-0.194461117065649,-0.372364689766463,-0.0916698497028211,4.28991596638655,13,-9750.764064780533,238,QUESTION_1,Klingon,3,3 +RESPONSE_23,5.52146091786225,-0.115441414536349,-0.189736659610103,-0.189736659610103,-0.154919333848297,4.6080000000000005,8,-6375.794678656474,250,QUESTION_1,Vulcan,3,3 +RESPONSE_24,5.56260286119342,-0.105870117611106,-0.107314094038344,-0.061957821080879,-0.0758825236114042,4.41458733205374,16,-5915.682845528709,521,QUESTION_1,Klingon,4,4 +RESPONSE_25,5.65248918026865,-0.112504894197645,-0.102597835208515,-0.0837707816583391,-0.0936585811581694,4.3754385964912315,8,-10498.89177526062,570,QUESTION_1,Esperanto,4,4 +RESPONSE_26,4.98360662170834,-0.0727538258830759,-0.154830884321333,-0.24828176658071,-0.0827605888602368,4.48972602739726,1,-11380.936100632374,292,QUESTION_1,Klingon,3,4 +RESPONSE_27,5.541263545158429,-0.0822912462783569,-0.0990147542976674,-0.276533159377486,-0.076696498884737,4.65490196078431,4,-5426.600365764597,510,QUESTION_1,Esperanto,4,4 +RESPONSE_28,5.95064255258773,-0.088600108396475,-0.169250800096582,-0.22821773229381895,-0.102062072615966,4.30989583333333,7,-10873.608949697917,384,QUESTION_1,Klingon,4,4 +RESPONSE_29,5.2704321630585005,-0.0947073041796312,-0.189709560179898,0.0,-0.101404025312679,4.45501285347044,3,-12268.102265407597,389,QUESTION_1,Esperanto,4,4 +RESPONSE_30,5.7021144175555,-0.0942795895294183,-0.0817177846445437,0.0,-0.0577831996656988,4.56093489148581,15,-9862.431539818015,599,QUESTION_1,Navi,4,4 +RESPONSE_31,5.105945473900579,-0.129148863101231,-0.239949489634293,-0.205971460217775,-0.14564381625088402,4.43030303030303,8,-11212.130837494527,330,QUESTION_1,Navi,4,3 +RESPONSE_32,5.48063892334199,-0.0676744054489706,-0.129099444873581,0.0,-0.0790569415042095,4.61666666666667,7,-9937.91837271977,480,QUESTION_1,Klingon,4,4 +RESPONSE_33,4.69134788222914,-0.14730224214631,-0.317675040241849,-0.253417014989599,-0.0677285461478596,4.3302752293578,5,-7662.169525540232,218,QUESTION_1,Esperanto,3,3 +RESPONSE_34,5.36597601502185,-0.107541814854325,-0.231815266445093,-0.199297832823702,-0.118400555694579,4.3411214953271005,6,-12717.773145363146,428,QUESTION_1,Esperanto,3,3 +RESPONSE_35,5.2094861528414205,-0.106270771263333,-0.295688508381829,-0.32221908170066804,-0.165294901226822,3.8551912568306,18,-7267.494103705405,366,QUESTION_1,Klingon,4,4 +RESPONSE_36,5.62040086571715,-0.182573854029988,-0.248181794051358,-0.4170288281141501,-0.120385853085769,4.21739130434783,11,-5217.811151300712,276,QUESTION_1,Esperanto,3,3 +RESPONSE_37,4.48300255201388,-0.0931767786938378,-0.22549380840084896,-0.260377821961648,-0.150329205600566,5.03389830508475,13,-10302.543876841462,177,QUESTION_1,Klingon,3,3 +RESPONSE_38,5.27128869416012,-0.114902876976361,-0.0925291512747007,-0.117041147196131,-0.101360606759923,4.56506849315068,18,-10041.230081425661,584,QUESTION_1,Klingon,4,4 +RESPONSE_39,5.257495372027781,-0.0942674190068453,-0.102062072615966,0.0,-0.14433756729740602,4.66927083333333,20,-9598.577946990195,384,QUESTION_1,Klingon,3,3 +RESPONSE_40,5.491001710377541,-0.103144669957614,-0.0786483757639614,-0.101534616513362,-0.0642161299067936,4.54432989690722,6,-9701.10455803817,485,QUESTION_1,Esperanto,5,4 +RESPONSE_41,5.01285505083457,-0.0696458972062817,-0.199778147463608,0.0,-0.0941763218696022,4.31929046563193,11,-12436.32145621167,451,QUESTION_1,Vulcan,3,3 +RESPONSE_42,5.0401940963378005,-0.0952674137029764,-0.17066403719657194,-0.3686764572048061,-0.127205462828696,4.62459546925566,10,-11555.641909086522,309,QUESTION_1,Vulcan,3,4 +RESPONSE_43,4.9416424226093,-0.0820436947691321,-0.253546276418555,0.0,-0.13363062095621198,4.29642857142857,10,-8758.190618056196,280,QUESTION_1,Esperanto,4,3 +RESPONSE_44,5.38449506278909,-0.10195650646170197,-0.20318563844357893,-0.279251949637145,-0.10708823421953,4.36238532110092,1,-9936.868277901563,436,QUESTION_1,Vulcan,4,3 +RESPONSE_45,3.17805383034795,-0.08602610524511099,-0.40824829046386296,-0.629152869605896,-0.14433756729740602,4.45833333333333,13,-7914.274147229711,48,QUESTION_1,Vulcan,1,1 +RESPONSE_46,5.0369526024136295,-0.0806323569636637,-0.21320071635561,-0.279145263119541,-0.113960576459638,4.0129870129870095,5,-9379.816307227742,308,QUESTION_1,Klingon,3,3 +RESPONSE_47,4.53259949315326,-0.0629442033841247,-0.179605302026775,-0.502680984365841,-0.12700012700019098,4.18279569892473,20,-8723.377843759363,186,QUESTION_1,Navi,3,3 +RESPONSE_48,5.161924741642482,-0.11768643872547198,-0.22070478408769895,-0.288261263221276,-0.11969397463728,4.36389684813754,12,-6322.170885920463,349,QUESTION_1,Klingon,4,4 +RESPONSE_49,5.87773578177964,-0.0584142836489996,-0.105851224804993,0.0,-0.074848118856512,4.624649859943982,11,-4917.104504391469,357,QUESTION_1,Klingon,5,4 +RESPONSE_50,4.95934199970871,-0.106059242110605,-0.236939551103637,0.0,-0.132453235706504,4.5649122807017495,17,-5984.200485280721,285,QUESTION_1,Vulcan,3,3 +RESPONSE_51,5.03370056702725,-0.0847759562482758,-0.235318067376639,-0.18928977492856894,-0.127619140222539,4.1693811074918585,14,-8588.600524316002,307,QUESTION_1,Esperanto,3,2 +RESPONSE_52,4.7791234931115305,-0.0809804323159801,-0.140028008402801,-0.158776837207489,0.0,4.55182072829132,19,-8068.081667857978,357,QUESTION_1,Esperanto,3,3 +RESPONSE_53,5.7868973813667095,-0.0809264248124579,-0.0783260449987957,0.0,-0.0553848775621711,4.5582822085889605,13,-8371.165051049893,652,QUESTION_1,Vulcan,5,4 +RESPONSE_54,5.38449506278909,-0.13188282274331098,-0.19156525704423005,0.0,-0.0957826285221151,4.39678899082569,8,-12814.529686809947,436,QUESTION_1,Navi,4,4 +RESPONSE_55,5.53338948872752,-0.120888152710087,-0.243492376778837,-0.251477845384773,0.0,4.80237154150198,10,-10310.61099250967,253,QUESTION_1,Vulcan,3,3 +RESPONSE_56,5.40941141405362,-0.0855687688669139,-0.141895130952121,0.0,-0.0668900062090651,4.8791946308724805,3,-6683.395797326215,447,QUESTION_1,Esperanto,4,4 +RESPONSE_57,4.67749084756772,-0.0889991364409946,-0.118124884643724,-0.15249857033260505,-0.0964485644340824,5.02325581395349,11,-8639.288449443839,215,QUESTION_1,Esperanto,3,3 +RESPONSE_58,5.4638318050256105,-0.0617948734672955,-0.130188910980824,0.0,-0.0650944554904119,4.46610169491525,3,-9074.087246915358,472,QUESTION_1,Vulcan,4,4 +RESPONSE_59,5.24174701505964,-0.10481038465151,-0.16265001215808894,-0.454256762579498,-0.154303349962092,4.03703703703704,10,-7906.506638398983,378,QUESTION_1,Klingon,3,3 +RESPONSE_60,5.105945473900579,-0.0963663112253405,-0.134839972492648,-0.325669473639465,-0.0953462589245592,4.57575757575758,20,-10414.510226552256,330,QUESTION_1,Vulcan,4,4 +RESPONSE_61,5.114995309420499,-0.114525687204238,-0.212238179989004,-0.305111607601371,-0.154996851658426,4.489489489489491,19,-8566.848384984332,333,QUESTION_1,Vulcan,3,4 +RESPONSE_62,5.325446033936291,-0.0974766470524472,-0.231361147419526,-0.26563059068111106,-0.04932636236669901,4.10948905109489,9,-9396.901884143404,411,QUESTION_1,Klingon,3,3 +RESPONSE_63,6.519147287940401,-0.0886330547993254,-0.0940720868383597,-0.127374207713181,-0.0768095372642568,4.572271386430679,12,-6275.554548944276,678,QUESTION_1,Klingon,6,6 +RESPONSE_64,5.96357934361845,-0.0810422683584207,-0.0620970299700831,0.0,-0.0620970299700831,4.64652956298201,15,-8797.091956370972,778,QUESTION_1,Klingon,4,4 +RESPONSE_65,3.9982007016692,-0.12166626259188,-0.287347885566345,0.0,-0.0957826285221151,5.08256880733945,14,-11345.69971346172,109,QUESTION_1,Navi,3,2 +RESPONSE_66,5.899897353582491,-0.0873868318784398,-0.261711961295107,-0.305205971323477,-0.104684784518043,4.4575342465753405,16,-7978.082993598017,365,QUESTION_1,Vulcan,4,4 +RESPONSE_67,5.170483995038151,-0.101581055955408,-0.106600358177805,0.0,-0.0753778361444409,4.63352272727273,1,-10978.98376463994,352,QUESTION_1,Vulcan,4,4 +RESPONSE_68,5.79909265446053,-0.0716422637964206,-0.0778498944161523,0.0,-0.0870388279778489,4.34393939393939,9,-6811.314110625066,660,QUESTION_1,Klingon,3,4 +RESPONSE_69,5.8006066592917405,-0.0727307034874147,-0.129001753263015,-0.19832887310274,-0.102907799163101,4.47957639939486,13,-9574.7074115321,661,QUESTION_1,Navi,4,4 +RESPONSE_70,4.981320906427481,-0.102095099381229,-0.287018923940964,0.0,-0.095672974646988,4.30663615560641,7,-9008.938820732785,437,QUESTION_1,Navi,4,3 +RESPONSE_71,4.26267987704132,-0.0972013601715729,-0.20555661294825894,-0.41959067914834497,-0.1876466562602,4.570422535211269,6,-4141.588654901808,142,QUESTION_1,Klingon,2,2 +RESPONSE_72,5.10897119481712,-0.14281029178319,-0.182298185396321,-0.31092882973458896,-0.109929941985863,4.66465256797583,10,-12447.35336294324,331,QUESTION_1,Klingon,4,4 +RESPONSE_73,5.105945473900579,-0.1331469260356,-0.233549683248457,-0.369274472937998,-0.0778498944161523,4.236363636363641,2,-10257.798364714119,330,QUESTION_1,Klingon,3,4 +RESPONSE_74,4.820281565605041,-0.140691853207243,-0.29784169859063503,-0.508000508000762,-0.179605302026775,4.84677419354839,6,-9806.210283716757,124,QUESTION_1,Vulcan,2,2 +RESPONSE_75,5.04985600724954,-0.093578853372847,-0.196116135138184,-0.325221817793995,-0.12659242088545802,4.06089743589744,5,-8531.132305254208,312,QUESTION_1,Klingon,3,3 +RESPONSE_76,5.28066243130932,-0.0674645775567244,-0.252216636152659,0.0,-0.0873704056661038,4.582697201017808,11,-8167.411227669309,393,QUESTION_1,Navi,4,4 +RESPONSE_77,5.9080829381689295,-0.0747561603981595,-0.0638442398069062,-0.0521286035142687,-0.0902893898143269,4.71195652173913,9,-8394.049034074327,736,QUESTION_1,Esperanto,4,5 +RESPONSE_78,5.2933048247244905,-0.13009636106238498,-0.187552337753985,-0.22416791983111,-0.15851065623706,4.59045226130653,10,-10501.899156105681,398,QUESTION_1,Esperanto,3,4 +RESPONSE_79,4.89034912822175,-0.0863779189997191,-0.173421993904824,-0.26726124191242395,-0.106198848810718,4.45488721804511,13,-8530.733733819932,266,QUESTION_1,Klingon,4,4 +RESPONSE_80,4.65078069724014,-0.12344473481452,-0.05643326479831,-0.319234753787049,-0.0798086884467622,4.582802547770701,18,-4005.124989343706,314,QUESTION_1,Esperanto,4,4 +RESPONSE_81,3.98898404656427,-0.250085554122964,-0.235702260395516,-0.52704627669473,-0.19245008972987496,3.2407407407407405,16,-4098.35574349779,54,QUESTION_1,Vulcan,1,1 +RESPONSE_82,5.220355825078321,-0.0733121425314626,-0.0900450337781496,-0.164398987305357,-0.0519875244910036,4.6972972972973,2,-11635.269177358607,370,QUESTION_1,Navi,4,4 +RESPONSE_83,5.1957307777729405,-0.0837175977383751,-0.189765856603368,-0.174559199492389,-0.128920512778062,4.28531855955679,20,-10909.455982109284,361,QUESTION_1,Klingon,4,4 +RESPONSE_84,5.41387570718231,-0.105431824122804,-0.2002225932798,0.0,-0.0943858356366017,4.2293986636971095,10,-10102.957727315257,449,QUESTION_1,Esperanto,4,4 +RESPONSE_85,5.21764946348058,-0.0958965176580497,-0.172656590094271,-0.0520579206295354,-0.073621017383231,4.62330623306233,12,-3802.700382709034,369,QUESTION_1,Navi,4,4 +RESPONSE_86,6.26530121273771,-0.0889230395342251,-0.15104200810677698,-0.272294852489016,-0.0436020720196947,4.3212927756654,2,-3011.5521246236267,526,QUESTION_1,Klingon,4,4 +RESPONSE_87,5.24174701505964,-0.0714053814475853,-0.21206967009345104,-0.205737799949456,-0.0890870806374748,4.26719576719577,17,-8516.635363424219,378,QUESTION_1,Klingon,3,3 +RESPONSE_88,6.08335981181224,-0.113088499743404,-0.147189558897955,-0.0955091976801456,-0.0955091976801456,4.72976054732041,8,-9200.17933048399,877,QUESTION_1,Vulcan,5,6 +RESPONSE_89,5.07829394257007,-0.071709912710422,-0.243290014494323,-0.17650112740455198,-0.0558145572185948,4.51401869158879,20,-7850.765137901687,321,QUESTION_1,Esperanto,3,3 +RESPONSE_90,5.19849703126583,-0.123592169420312,-0.22298824387415,-0.37164707312358297,-0.14865882924943302,4.425414364640879,18,-7923.227919310027,181,QUESTION_1,Vulcan,3,3 +RESPONSE_91,5.59842195899838,-0.105539610082724,-0.0608580619450185,0.0,-0.0608580619450185,4.7148148148148215,6,-12922.833953692101,540,QUESTION_1,Vulcan,4,4 +RESPONSE_92,5.30081424674662,-0.103164623391988,-0.122321680483099,-0.27804088398486,-0.0706224551546449,4.38403990024938,19,-6263.034033573475,401,QUESTION_1,Esperanto,4,4 +RESPONSE_93,5.39362754635236,-0.12197228901347,-0.190692517849118,-0.286038776773678,-0.116774841624228,4.22727272727273,11,-11072.920394610806,220,QUESTION_1,Vulcan,2,3 +RESPONSE_94,5.7120812774709,-0.0671150404290653,-0.0995859195463938,-0.253895273523072,-0.0909090909090909,4.30743801652893,9,-8809.405249491232,605,QUESTION_1,Vulcan,4,4 +RESPONSE_95,4.48863636973214,-0.136637473425349,-0.224859506698758,-0.3671940368172629,-0.14990633779917198,4.08426966292135,9,-6588.630126692189,178,QUESTION_1,Navi,1,2 +RESPONSE_96,5.2230548820474905,-0.119586510606659,-0.32835455165155897,-0.315801295470782,-0.155752239495349,3.91374663072776,19,-10052.42065009408,371,QUESTION_1,Klingon,3,3 +RESPONSE_97,5.07517381523383,-0.0926733451834833,-0.185404962177392,-0.285043856274785,-0.125,4.528125,19,-8407.861270281739,320,QUESTION_1,Klingon,3,3 +RESPONSE_98,5.17897060891547,-0.105064980558513,-0.265372446217138,-0.359968699734758,-0.0750586625040802,3.87323943661972,15,-9684.254227582018,355,QUESTION_1,Klingon,4,4 +RESPONSE_99,5.5797298259862185,-0.093605282305646,-0.0752354793863279,-0.225706438158984,-0.0971285862357264,4.02075471698113,14,-9259.886331494963,530,QUESTION_1,Klingon,3,3 +RESPONSE_100,4.59511985013459,-0.113772247737636,-0.201007563051842,-0.201007563051842,-0.100503781525921,4.77777777777778,5,-12161.500227998526,198,QUESTION_1,Vulcan,4,3 +RESPONSE_101,4.918788616166248,-0.05846631365561208,-0.08367329524923198,-0.31640105883090647,-0.06052478986940827,4.653360694245869,6,-6465.106227884413,279,QUESTION_2,Vulcan,3,3 +RESPONSE_102,5.37747151573321,-0.07684403613744652,-0.1587168205942022,-0.27014893947230545,-0.1051067400115647,4.724949913148605,17,-4519.996829437504,434,QUESTION_2,Navi,4,4 +RESPONSE_103,4.892111017143179,-0.06051433741639946,-0.1392801875175425,-0.3006749735095473,0.0015104495451854455,4.654320811675878,10,-11123.705001823499,261,QUESTION_2,Navi,3,3 +RESPONSE_104,4.723079342433,-0.052497444510860476,-0.20901104404686718,-0.3078493125618784,-0.0667171933895866,4.397242437123226,3,-9264.340829474831,224,QUESTION_2,Klingon,3,3 +RESPONSE_105,4.645966352068384,-0.11680791540840989,-0.12082130659215135,-0.2583245876793123,-0.12156465077973345,4.975783199375207,8,-7398.701061176511,204,QUESTION_2,Esperanto,3,3 +RESPONSE_106,5.465065975875674,-0.06166209968424947,-0.20435740079826673,-0.0015795749069711772,-0.10123606943084622,4.526371794827584,6,-12571.634915790935,504,QUESTION_2,Klingon,4,4 +RESPONSE_107,4.965706929020117,-0.07254350596760864,-0.11363941547042211,-0.18025455015829475,-0.056176790322868035,4.57340143046316,15,-7087.393198740861,314,QUESTION_2,Navi,3,3 +RESPONSE_108,4.7205689818849885,-0.10761083254717287,-0.20675884772825343,-0.06361508954095704,-0.14396084781532367,4.99523329792926,8,-6478.817061482429,237,QUESTION_2,Klingon,3,4 +RESPONSE_109,5.434386814142545,-0.11954882354656479,-0.25986529451298285,-0.22559366203569522,-0.1135873968561824,3.8715348644997283,19,-9658.401584608222,474,QUESTION_2,Klingon,3,3 +RESPONSE_110,6.285270322709768,-0.11089498366460575,-0.33254963880103205,-0.001375991250069592,-0.061761109203762016,4.24629834604931,5,-11126.400910211223,528,QUESTION_2,Esperanto,2,2 +RESPONSE_111,6.165592319533938,-0.08216676605907383,-0.14546022836917108,0.00022155959084697825,-0.12133332625889169,4.190146504313621,19,-8468.626667150322,469,QUESTION_2,Vulcan,3,3 +RESPONSE_112,5.3084535528107555,-0.0776359334934712,-0.19087656570446143,0.0005489639251680889,-0.050265964233439764,4.543903313402335,7,-8161.356670511145,397,QUESTION_2,Navi,4,4 +RESPONSE_113,5.2317638420281,-0.06986488656719946,-0.19868133918374636,-0.15877996138588668,-0.07557467590174678,4.418046675506115,3,-5211.347773253591,356,QUESTION_2,Navi,3,3 +RESPONSE_114,5.458551243374556,-0.11508142277984058,-0.06470852270318823,-0.0007958938323651664,-0.08929299612293681,4.567467105308518,17,-11991.386107675116,484,QUESTION_2,Esperanto,4,4 +RESPONSE_115,4.4241886512012405,-0.15772389901637127,-0.38520105939903954,-0.35608547175852845,-0.07711472431028461,4.864837706985233,15,-10760.800223719805,170,QUESTION_2,Klingon,2,2 +RESPONSE_116,5.311088130806796,-0.1135703731825288,-0.14938408631686176,-0.1336804254344433,-0.07021542150959034,4.036380262814226,8,-5823.56093973633,405,QUESTION_2,Klingon,3,4 +RESPONSE_117,6.096948536400686,-0.07568647149040199,-0.28021969252545,-9.465642235259125e-05,-0.1606521037073779,3.691675184197831,9,-6926.527679268197,493,QUESTION_2,Navi,3,3 +RESPONSE_118,5.574328182636037,-0.0673365939164496,-0.07058295282322744,-0.1690486263700711,-0.0005432087480468886,4.863153780686836,5,-9826.941328268971,593,QUESTION_2,Navi,4,5 +RESPONSE_119,4.5566511729620105,-0.0579699825963668,-0.19460498381258728,-0.3126722018887331,0.0003669073466470883,4.196546196520618,13,-7036.059740115639,185,QUESTION_2,Vulcan,3,3 +RESPONSE_120,4.72914074340382,-0.15263255406282358,-0.3079656228658358,-0.18181113120381237,-0.1302246603401582,4.46891340069981,10,-4898.643831744854,232,QUESTION_2,Esperanto,3,3 +RESPONSE_121,4.609700108773976,-0.09517717030677017,-0.18676449462537073,-0.0002503602138092688,0.0009047218845006269,4.581092873870787,3,-9004.720335548493,196,QUESTION_2,Vulcan,3,3 +RESPONSE_122,4.852089316410692,-0.12340034657548192,-0.19464534606405887,-0.3703551443452912,-0.0914826491457294,4.280752782612603,11,-4027.976692812015,238,QUESTION_2,Vulcan,3,3 +RESPONSE_123,5.340511992805594,-0.11542384521725352,-0.18763352076322767,-0.18781700322968817,-0.15737035225116927,4.552739091642029,14,-4156.727664734305,250,QUESTION_2,Navi,3,3 +RESPONSE_124,5.459774773852938,-0.10590875061627138,-0.10799159818250463,-0.06275129400236999,-0.07612400968834564,4.417949622052099,9,-12956.343742644221,521,QUESTION_2,Vulcan,4,4 +RESPONSE_125,5.690480969731521,-0.11394322884434634,-0.10273127766056726,-0.08358453508573804,-0.09311446464940752,4.337386389743431,6,-9576.489599905985,570,QUESTION_2,Vulcan,4,4 +RESPONSE_126,4.909202408902895,-0.07282056633210035,-0.15623290200622897,-0.2481733446968701,-0.08199439043721442,4.506021905259693,11,-10009.440400150155,292,QUESTION_2,Vulcan,3,4 +RESPONSE_127,5.495426320310753,-0.0826597617148281,-0.0979165568731384,-0.27948639842149836,-0.07660464585528054,4.671329360015528,20,-5351.445829058988,510,QUESTION_2,Klingon,4,4 +RESPONSE_128,6.005043061379481,-0.08886397856099729,-0.16992430861096935,-0.22740158945927197,-0.10153481957799744,4.308450052502083,13,-11971.647281941385,384,QUESTION_2,Klingon,4,4 +RESPONSE_129,5.221355857926471,-0.09518674048789358,-0.19145816094012535,0.0004098307160566024,-0.10140473599301686,4.4754524132542075,3,-2680.4031019342965,389,QUESTION_2,Navi,4,4 +RESPONSE_130,5.705062634038631,-0.09437861298765658,-0.0806167248880774,0.0004303060776907982,-0.0579158456945334,4.612494095393959,14,-9880.79310468816,599,QUESTION_2,Vulcan,4,4 +RESPONSE_131,5.13800427314506,-0.12980386937886454,-0.23734198023562647,-0.2086828919376892,-0.1479918187970316,4.458264617823093,6,-8492.810181984578,330,QUESTION_2,Klingon,4,3 +RESPONSE_132,5.499781524342485,-0.06733253474067233,-0.12940789438889239,0.0011751540218018423,-0.07842791433964605,4.625524215343247,15,-6698.431350245865,480,QUESTION_2,Klingon,4,4 +RESPONSE_133,4.695492661334901,-0.14604211966708427,-0.31556646178034703,-0.25738480891728005,-0.06797648233386815,4.318492394915499,5,-9894.40061913526,218,QUESTION_2,Vulcan,3,3 +RESPONSE_134,5.366618894820827,-0.10734273342349229,-0.2315527711914587,-0.1987574951615901,-0.12216754117278233,4.2324548492921865,11,-11927.210523855365,428,QUESTION_2,Esperanto,3,3 +RESPONSE_135,5.248529985824587,-0.10504103193597472,-0.2960233043259136,-0.32273290915690594,-0.16360317895827797,3.8114005791646837,4,-3711.225092608431,366,QUESTION_2,Vulcan,4,4 +RESPONSE_136,5.616205456265176,-0.184531489267392,-0.24562883317963574,-0.4162640011027068,-0.1201003293746354,4.258145607381502,4,-5573.095885703102,276,QUESTION_2,Vulcan,3,3 +RESPONSE_137,4.47883905437411,-0.09192774917116499,-0.22352809645636212,-0.2569382119134297,-0.14959977335288158,5.031419096405642,11,-7967.397763617708,177,QUESTION_2,Klingon,3,3 +RESPONSE_138,5.334805432864205,-0.11327385659262186,-0.09409682719372153,-0.11781165823623599,-0.10077403363121103,4.658144717155632,14,-9311.575277488268,584,QUESTION_2,Navi,4,4 +RESPONSE_139,5.24787683761572,-0.09376094123741294,-0.10178086362512657,0.0009161997944107991,-0.141196734467227,4.639446939442154,10,-6039.456545370538,384,QUESTION_2,Navi,3,3 +RESPONSE_140,5.505760447005617,-0.10228978538035621,-0.07672939617100236,-0.1030870575612911,-0.06361958943206988,4.574385568635944,14,-7957.536410547242,485,QUESTION_2,Vulcan,5,4 +RESPONSE_141,5.067414355527589,-0.07032841654776159,-0.20125146583498082,0.0004209907885933485,-0.09374082709815304,4.306128523239099,2,-4805.879179727498,451,QUESTION_2,Klingon,3,3 +RESPONSE_142,4.990222856346401,-0.09572962930799064,-0.16829708334080418,-0.3698460109843676,-0.12512912850886954,4.70020105441579,20,-6113.761458176679,309,QUESTION_2,Navi,3,4 +RESPONSE_143,4.972380209054976,-0.08259455352140707,-0.2535209038529035,-0.0003938195059140791,-0.13295351081667459,4.328411264073803,3,-7277.182650695792,280,QUESTION_2,Vulcan,4,3 +RESPONSE_144,5.2953664800958,-0.10066855463219503,-0.204688974613218,-0.2732285432873661,-0.1057157132251066,4.3553366865861545,1,-11018.53696777083,436,QUESTION_2,Navi,4,3 +RESPONSE_145,3.171619158835189,-0.08582372653070013,-0.4109828618814131,-0.6252342229301503,-0.14445996188951193,4.493463026954399,15,-4664.820098801805,48,QUESTION_2,Navi,1,1 +RESPONSE_146,5.064239928501901,-0.08097278514831409,-0.21297534167206753,-0.2805296780270174,-0.11535899873783596,3.9240371613480547,13,-12744.182391698272,308,QUESTION_2,Navi,3,3 +RESPONSE_147,4.534629133303021,-0.06313446659749393,-0.17981159034889505,-0.5030389548419967,-0.1279700607761672,4.176272265325383,15,-3859.9569902010103,186,QUESTION_2,Klingon,3,3 +RESPONSE_148,5.107959750641561,-0.11913395918418351,-0.21874767689530272,-0.2885406632576389,-0.12026526526352894,4.381008466336507,7,-9249.949334429717,349,QUESTION_2,Klingon,4,4 +RESPONSE_149,5.8147372876361,-0.058623441121834034,-0.10552540568012526,0.0005189589910037854,-0.07467483381847936,4.634307335358474,15,-12169.929838349979,357,QUESTION_2,Vulcan,5,4 +RESPONSE_150,5.011928849903473,-0.10619220833344621,-0.23867664181960788,-0.0014195998589563144,-0.13323905662341679,4.515304250971422,16,-8076.56422492977,285,QUESTION_2,Klingon,3,3 +RESPONSE_151,4.995261279543159,-0.08490811901022546,-0.2414627750209928,-0.18916902982073774,-0.13104733924742903,4.088903689487477,13,-9312.417218508825,307,QUESTION_2,Navi,3,2 +RESPONSE_152,4.8201758746658125,-0.08176919877724853,-0.13993645446446504,-0.15874987908147856,0.0011701317813325098,4.5417198524390106,7,-13039.579579690433,357,QUESTION_2,Navi,3,3 +RESPONSE_153,5.807844464457312,-0.08114751869850267,-0.0789357146716636,0.00048036521113192516,-0.05595580562126117,4.570788122075371,20,-8399.828719692361,652,QUESTION_2,Klingon,5,4 +RESPONSE_154,5.396123090091687,-0.13365118832927428,-0.19052850850740688,-0.0005183264021259889,-0.09666057297380977,4.41871814084749,14,-11445.916331723058,436,QUESTION_2,Vulcan,4,4 +RESPONSE_155,5.531690065783789,-0.12134638031335492,-0.2470197084981346,-0.2525020404195863,0.0007386532484792824,4.747924320770324,14,-5879.54293838799,253,QUESTION_2,Esperanto,3,3 +RESPONSE_156,5.322665659196558,-0.08457760452567545,-0.14291443474210014,-0.0010100192232039588,-0.06709954938272918,4.958268048793898,20,-4746.35643936117,447,QUESTION_2,Vulcan,4,4 +RESPONSE_157,4.711074921103684,-0.08939026663745393,-0.11921224913123171,-0.15123338146577328,-0.09634965008730112,4.9200356286196865,17,-11289.29224878837,215,QUESTION_2,Esperanto,3,3 +RESPONSE_158,5.476959380443143,-0.06245215085946164,-0.13090113052138827,-0.0006096558563298874,-0.06475928078748111,4.46242592210612,17,-8504.559803174887,472,QUESTION_2,Klingon,4,4 +RESPONSE_159,5.232870864503079,-0.10364059676750166,-0.1589175691327785,-0.4622530718501322,-0.15347948092283792,4.00364050044325,4,-11292.37987847025,378,QUESTION_2,Navi,3,3 +RESPONSE_160,5.109580142712317,-0.0947832754520689,-0.13525602158356728,-0.32061253948683144,-0.09557559744101637,4.5484384729973755,18,-11880.315902354852,330,QUESTION_2,Navi,4,4 +RESPONSE_161,5.093521735274523,-0.11638205021904752,-0.21018061620149292,-0.30795448317823304,-0.1567645156773073,4.5202553242247,7,-10324.86783250445,333,QUESTION_2,Vulcan,3,4 +RESPONSE_162,5.3827927773647515,-0.09879977099601892,-0.23374409281452235,-0.267218104446801,-0.05020657637175471,4.0787091853407285,3,-11064.976037479895,411,QUESTION_2,Vulcan,3,3 +RESPONSE_163,6.517289427298536,-0.08873233766135251,-0.0948112327205807,-0.12695360134207034,-0.07672343442969053,4.572380297545303,5,-6759.5634949097985,678,QUESTION_2,Navi,6,6 +RESPONSE_164,6.007359193895808,-0.08160059759801111,-0.06322410436987919,-0.0005548139642296516,-0.06170714331491424,4.678286147948574,15,-4863.5602308434,778,QUESTION_2,Navi,4,4 +RESPONSE_165,3.9947763822980913,-0.12052702430231665,-0.289705118097749,0.0003497713158202486,-0.09541616145055924,5.019889252325149,18,-3817.7415334896746,109,QUESTION_2,Vulcan,3,2 +RESPONSE_166,5.894626263346283,-0.08671605671664033,-0.26042245657154844,-0.3058988095529482,-0.10598510523329266,4.515923628782843,6,-11927.052568482386,365,QUESTION_2,Esperanto,4,4 +RESPONSE_167,5.224662748262889,-0.10122505176379183,-0.1044392837630142,0.0008025123804133728,-0.07497108681623603,4.651259864473101,5,-7835.144404166324,352,QUESTION_2,Navi,4,4 +RESPONSE_168,5.844278803112625,-0.07185373844892601,-0.0785893542835399,-0.0003679250686023668,-0.08619034002037293,4.309871593022227,12,-7809.517251530452,660,QUESTION_2,Navi,3,4 +RESPONSE_169,5.7379266283433035,-0.07309350722723108,-0.129273086499849,-0.19955461921407305,-0.10430746669332096,4.464918710228102,19,-7200.351976830805,661,QUESTION_2,Esperanto,4,4 +RESPONSE_170,4.9701724892605865,-0.10245634580316637,-0.2852246298807202,-0.0013397015006786325,-0.09611443062517884,4.303402662176674,19,-11847.462673468022,437,QUESTION_2,Esperanto,4,3 +RESPONSE_171,4.291948467390473,-0.09635777257971156,-0.2045636791298859,-0.4150377849247746,-0.18661507548223152,4.725449205476492,19,-7985.727976371693,142,QUESTION_2,Esperanto,2,2 +RESPONSE_172,5.071358755167781,-0.1434575189689885,-0.18029291584301013,-0.301894946149314,-0.10945441992926454,4.6680260288259925,20,-3439.592062829312,331,QUESTION_2,Vulcan,4,4 +RESPONSE_173,5.1051763639541425,-0.1337724184817774,-0.23124132831009803,-0.3758469205336675,-0.0797552178491058,4.248915006706539,2,-5240.4860943759995,330,QUESTION_2,Vulcan,3,4 +RESPONSE_174,4.787539684979566,-0.13926892023419327,-0.29518644491506346,-0.5150133496826935,-0.17816115654205306,4.7026944557647665,2,-8430.822103326573,124,QUESTION_2,Navi,2,2 +RESPONSE_175,5.073186004643178,-0.09337400497670467,-0.19605250446101183,-0.32858817782204275,-0.12703148754313848,4.057181113391399,8,-7974.097539195111,312,QUESTION_2,Esperanto,3,3 +RESPONSE_176,5.221162796404406,-0.06853762794234164,-0.2523978501508127,-0.00169050687658027,-0.08622080644231142,4.579923979700583,7,-9883.802826891471,393,QUESTION_2,Klingon,4,4 +RESPONSE_177,6.033321810861046,-0.07448272350519143,-0.06301784898188022,-0.05328240694063912,-0.09143767127822963,4.737753488622302,5,-11994.672423336013,736,QUESTION_2,Esperanto,4,5 +RESPONSE_178,5.3790174720294335,-0.1271026470980864,-0.19328909407765843,-0.22598664565756277,-0.16010846497436462,4.628596178480991,16,-5611.522716602072,398,QUESTION_2,Navi,3,4 +RESPONSE_179,4.92041940068506,-0.08749274336110036,-0.17421562697494125,-0.26509829951636277,-0.10516753300107742,4.531233134954746,15,-9491.096843062174,266,QUESTION_2,Navi,4,4 +RESPONSE_180,4.609519700655417,-0.12495481704631227,-0.05658587327314981,-0.31816334868478263,-0.07970551697902148,4.5548830077927756,10,-10608.465468523282,314,QUESTION_2,Klingon,4,4 +RESPONSE_181,3.9466557731238225,-0.252006351341356,-0.23666574929665557,-0.5363895117926175,-0.1937102857960339,3.2125821261868217,13,-9771.27033469045,54,QUESTION_2,Esperanto,1,1 +RESPONSE_182,5.265720578587766,-0.07477700842616553,-0.08973708206445748,-0.1614557188668088,-0.05110129600414879,4.726341126279555,16,-8732.319959541193,370,QUESTION_2,Klingon,4,4 +RESPONSE_183,5.199164262406118,-0.08216105459854368,-0.18929193749535597,-0.17349931156634626,-0.1294153915446259,4.168862408279101,6,-9549.850791445086,361,QUESTION_2,Klingon,4,4 +RESPONSE_184,5.459848959333718,-0.10410630971407253,-0.19981987215922487,-0.00040164324516208316,-0.09388902388476827,4.2555126510180035,14,-7955.043721243641,449,QUESTION_2,Navi,4,4 +RESPONSE_185,5.119145271597595,-0.09566613707870053,-0.17115881539644967,-0.05190210141972036,-0.07393262379021222,4.623614264112992,18,-11785.844061635773,369,QUESTION_2,Vulcan,4,4 +RESPONSE_186,6.3039870125148445,-0.0891120259797746,-0.15104226062587708,-0.2756254164535808,-0.04357722171701301,4.3025482004414775,7,-6333.675210948576,526,QUESTION_2,Navi,4,4 +RESPONSE_187,5.175686861027831,-0.07200301813728856,-0.21486380570872984,-0.20380667506964112,-0.09043609134289335,4.232803345609543,19,-8273.83374046214,378,QUESTION_2,Vulcan,3,3 +RESPONSE_188,6.009586601169855,-0.11248424275991507,-0.14775646026778888,-0.09462448536671209,-0.09676477400103128,4.749416079421529,3,-7420.193065714459,877,QUESTION_2,Vulcan,5,6 +RESPONSE_189,5.119904434545348,-0.07244798579967264,-0.24614172887587013,-0.17817268938751685,-0.05651017820094121,4.503211568353783,12,-7425.967000402626,321,QUESTION_2,Vulcan,3,3 +RESPONSE_190,5.293631262183446,-0.12305841161280363,-0.2192407107422868,-0.3725314758360176,-0.15067767815439584,4.429379774977941,1,-5142.562394178464,181,QUESTION_2,Esperanto,3,3 +RESPONSE_191,5.601878763028315,-0.10639041544370338,-0.06153711770890188,0.00020142096296177803,-0.06082476482949918,4.767709673282672,9,-9203.213790866088,540,QUESTION_2,Esperanto,4,4 +RESPONSE_192,5.266582941363001,-0.10444571426188826,-0.11983461031190364,-0.2736553580392473,-0.07139893882287071,4.3745684685045045,5,-4176.664313374371,401,QUESTION_2,Esperanto,4,4 +RESPONSE_193,5.392298986171343,-0.12199290087924103,-0.1896272304451308,-0.2840802474545391,-0.11964910089435968,4.192302958899154,18,-4528.737609707758,220,QUESTION_2,Vulcan,2,3 +RESPONSE_194,5.715903626051941,-0.06720619150629818,-0.0994711839474234,-0.249941723230415,-0.09142394657439383,4.300713055616408,17,-10329.666544679281,605,QUESTION_2,Vulcan,4,4 +RESPONSE_195,4.441954917708253,-0.13579044647686456,-0.22283940243135292,-0.3646157171147711,-0.1498651804507545,4.167608920931242,7,-7690.914934129363,178,QUESTION_2,Vulcan,1,2 +RESPONSE_196,5.209682498540727,-0.11833294965021894,-0.32796642773716594,-0.31619586715316206,-0.154551540956965,3.90703242212857,11,-10212.655046809434,371,QUESTION_2,Klingon,3,3 +RESPONSE_197,5.06337966988575,-0.09060774371971983,-0.18387059455920907,-0.2849963808375897,-0.12474602338655749,4.480911614442209,9,-10787.76586568092,320,QUESTION_2,Esperanto,3,3 +RESPONSE_198,5.126789743933638,-0.10342338044575332,-0.259430364206437,-0.35934962533735537,-0.07493499490432515,3.8148668704909827,19,-4391.695537621622,355,QUESTION_2,Esperanto,4,4 +RESPONSE_199,5.525661768985705,-0.09357990058828028,-0.07478428788331598,-0.22370751289848514,-0.09733630655618372,3.898900946289304,13,-9048.16811316891,530,QUESTION_2,Esperanto,3,3 +RESPONSE_200,4.609340569769084,-0.11390983303106346,-0.19995532761473275,-0.20189541778000183,-0.09891697581984626,4.784332523626739,8,-12696.37397988159,198,QUESTION_2,Vulcan,4,3 +RESPONSE_201,4.890696833978422,-0.058058724688164226,-0.08363529242132459,-0.31490192141786555,-0.05923644851680974,4.64627157747443,10,-6949.160151757958,279,QUESTION_3,Vulcan,3,3 +RESPONSE_202,5.37563634288815,-0.07753313894426579,-0.15845446214739115,-0.27329540365990757,-0.10698003700885667,4.7679685027649725,16,-10860.59788162963,434,QUESTION_3,Vulcan,4,4 +RESPONSE_203,4.86156766555642,-0.06043275207695878,-0.13914140623440255,-0.30895515968227505,-0.0015353352736332349,4.696743962584516,13,-9352.29985381608,261,QUESTION_3,Esperanto,3,3 +RESPONSE_204,4.715991499461418,-0.052555116725117305,-0.21426394754402558,-0.3075183864195222,-0.06720243303645106,4.43633135750083,18,-8047.206767822946,224,QUESTION_3,Klingon,3,3 +RESPONSE_205,4.669141375990851,-0.11758075649847768,-0.11990938069338256,-0.258564468842952,-0.1203433639835142,4.927806864757849,2,-8104.421869376339,204,QUESTION_3,Klingon,3,3 +RESPONSE_206,5.522489428422048,-0.06102659977546546,-0.2122510720391036,-0.0006529947782917726,-0.09893774696645952,4.569347858726433,11,-10953.368176939812,504,QUESTION_3,Vulcan,4,4 +RESPONSE_207,5.019397652241924,-0.07171343174749982,-0.11368490605615551,-0.17895461814475042,-0.055486683854997665,4.563519345963111,12,-8954.177088045732,314,QUESTION_3,Klingon,3,3 +RESPONSE_208,4.813184914505544,-0.10830266078132371,-0.2060405346137944,-0.06511654693941007,-0.14678487945830912,4.8807311859270035,6,-8384.90143616932,237,QUESTION_3,Esperanto,3,4 +RESPONSE_209,5.418701729003941,-0.11910797471578038,-0.2624014257951284,-0.22580039042448807,-0.11535985101775705,3.8745885960637416,18,-10741.382443952694,474,QUESTION_3,Esperanto,3,3 +RESPONSE_210,6.339975616815192,-0.11102050667910016,-0.32724961420810855,6.173785357909477e-07,-0.06105730063682494,4.25449402316225,7,-5973.248528457397,528,QUESTION_3,Esperanto,2,2 +RESPONSE_211,6.110503407121785,-0.07959244846959866,-0.1476098733864248,-0.0013093915639505714,-0.12429655218231966,4.195978904086468,19,-7875.505228004752,469,QUESTION_3,Esperanto,3,3 +RESPONSE_212,5.290492208360585,-0.07754627120662798,-0.18774826564090627,-0.001232270138544389,-0.050816220792918226,4.581794392760274,10,-10147.21649325098,397,QUESTION_3,Vulcan,4,4 +RESPONSE_213,5.097008453572262,-0.06978744556149652,-0.1988204339453971,-0.16445276873839224,-0.07503031906376244,4.474569426627528,3,-9895.303806981794,356,QUESTION_3,Vulcan,3,3 +RESPONSE_214,5.523022863276814,-0.11383293710339659,-0.064514517746239,0.002084214656098666,-0.08980606828501381,4.5511454308262005,11,-7155.183494920101,484,QUESTION_3,Vulcan,4,4 +RESPONSE_215,4.4118148451932715,-0.15818080047885796,-0.3915565724280131,-0.35546898032723223,-0.07760251234738437,4.760617491969533,1,-8692.832183500532,170,QUESTION_3,Navi,2,2 +RESPONSE_216,5.290367930371708,-0.11220806382383587,-0.14761897658097825,-0.13222134904589314,-0.07006200357539036,4.007840392854947,5,-5416.107607993237,405,QUESTION_3,Vulcan,3,4 +RESPONSE_217,6.193151535358735,-0.07744877325219555,-0.28003889146165484,0.00021475532579491274,-0.1593930640064699,3.674762065547799,3,-7534.377831513271,493,QUESTION_3,Klingon,3,3 +RESPONSE_218,5.746548043309294,-0.06794382905713074,-0.0700180342218257,-0.17134214201638948,0.002238968062144306,4.807994669139133,4,-5169.592032732934,593,QUESTION_3,Navi,4,5 +RESPONSE_219,4.576947347045769,-0.05882160641003464,-0.19169135669264978,-0.3083801993865496,0.0022326972907747065,4.178195989789485,8,-6159.9563962452485,185,QUESTION_3,Klingon,3,3 +RESPONSE_220,4.779064680386426,-0.1552276964477683,-0.30540442576274546,-0.1860150402757342,-0.13115009788167034,4.438644479692342,1,-9667.050508782597,232,QUESTION_3,Esperanto,3,3 +RESPONSE_221,4.539763600413071,-0.0963677983160747,-0.18592655906252734,0.0005187484561353012,-0.0019252460091651556,4.6256866940692225,9,-8559.58222877207,196,QUESTION_3,Klingon,3,3 +RESPONSE_222,4.733997482971051,-0.12414331702460948,-0.1944507292461859,-0.3662333389560549,-0.09123594092099274,4.314354041615088,13,-4117.449565674796,238,QUESTION_3,Klingon,3,3 +RESPONSE_223,5.4506917146065685,-0.11568458210035279,-0.18931543061460654,-0.19198199911525052,-0.15442880731205305,4.656730975564951,10,-6605.213842926654,250,QUESTION_3,Navi,3,3 +RESPONSE_224,5.458682127764484,-0.10397179239788273,-0.10599216550540196,-0.06053594386443343,-0.07480773541720409,4.423950949676425,11,-10996.160548652504,521,QUESTION_3,Vulcan,4,4 +RESPONSE_225,5.626536211506162,-0.11174936214768895,-0.10025565651626955,-0.08435937902875315,-0.09530530889577844,4.399148635068395,16,-10947.561136767334,570,QUESTION_3,Navi,4,4 +RESPONSE_226,4.980942046263076,-0.07246529681069656,-0.15306894908803476,-0.24795038042248768,-0.0834745022421755,4.423622353160669,16,-11011.561506874601,292,QUESTION_3,Navi,3,4 +RESPONSE_227,5.535614044643602,-0.08207322817907865,-0.10030858839907184,-0.2765686802101736,-0.07644188314653552,4.6533909062357575,12,-9194.05429103279,510,QUESTION_3,Esperanto,4,4 +RESPONSE_228,5.942796787095934,-0.08845689772101628,-0.1705135005817025,-0.22867048938715726,-0.1030357034760525,4.228981295342566,1,-8685.154594493553,384,QUESTION_3,Navi,4,4 +RESPONSE_229,5.305404353832902,-0.09523708553189954,-0.1890905455114685,-0.0006479786697715358,-0.10085489612737586,4.50534983566215,13,-6790.555677342784,389,QUESTION_3,Navi,4,4 +RESPONSE_230,5.6715938077879855,-0.09352360456930696,-0.0829008877930817,0.0003815118172626976,-0.058686599517734646,4.635254666652049,1,-5216.079975457344,599,QUESTION_3,Esperanto,4,4 +RESPONSE_231,5.173604900849305,-0.1275261560629917,-0.2421181934475855,-0.20436136868525814,-0.1476605755733995,4.385175216990089,6,-9056.903643062418,330,QUESTION_3,Klingon,4,3 +RESPONSE_232,5.498056212470209,-0.06818823730000227,-0.1291566102872696,0.0014828954297816772,-0.0788563732062439,4.669280081026077,20,-8436.13160060937,480,QUESTION_3,Navi,4,4 +RESPONSE_233,4.729385244687233,-0.14811494359403587,-0.32216828890803595,-0.25012603977046716,-0.06741215767276686,4.251617253049309,10,-9031.632863090246,218,QUESTION_3,Esperanto,3,3 +RESPONSE_234,5.229105526641137,-0.10621676921903953,-0.23166457254208928,-0.20032580150275972,-0.11820452802953609,4.319655922437745,8,-6060.272889419724,428,QUESTION_3,Vulcan,3,3 +RESPONSE_235,5.258759805511047,-0.1065078759306194,-0.3009692936383166,-0.3228969531710698,-0.16575282931714108,3.8313551953420566,7,-6965.098312326875,366,QUESTION_3,Vulcan,4,4 +RESPONSE_236,5.548740058621294,-0.1842457970026751,-0.24857397649968027,-0.4206322126062014,-0.12048788218709879,4.213750916711144,19,-11935.708899438774,276,QUESTION_3,Navi,3,3 +RESPONSE_237,4.459391361473164,-0.09394002341405368,-0.2282217265669959,-0.26056485205421603,-0.15146426353087875,4.977109886819933,4,-5987.172256236062,177,QUESTION_3,Vulcan,3,3 +RESPONSE_238,5.298964410022213,-0.11266774782590405,-0.09421391783084053,-0.11613614749542352,-0.10140744684666153,4.5752222921583465,16,-8712.847504373267,584,QUESTION_3,Vulcan,4,4 +RESPONSE_239,5.313119955810195,-0.09451488599554919,-0.10172224949109811,-0.0022561975058891285,-0.14151453339945938,4.632068887974614,13,-7986.477743061776,384,QUESTION_3,Vulcan,3,3 +RESPONSE_240,5.5151688743497,-0.10458043103435946,-0.07826920836126071,-0.1015685030182735,-0.06422100714311205,4.551288865137509,17,-5877.598667201126,485,QUESTION_3,Esperanto,5,4 +RESPONSE_241,5.137450517848309,-0.06993794399189673,-0.19795919850871618,0.0005014847292509028,-0.09122985844026311,4.368022034434312,17,-9233.753380028009,451,QUESTION_3,Vulcan,3,3 +RESPONSE_242,4.993152803826558,-0.09502796240890307,-0.170478223748263,-0.37144618745429714,-0.12745026948768465,4.618099882347924,3,-5204.423126809664,309,QUESTION_3,Vulcan,3,4 +RESPONSE_243,4.937001088124836,-0.08104012184524384,-0.2537171101446886,-0.0004954277080145841,-0.13172897919211685,4.246024281317495,18,-8879.042434081259,280,QUESTION_3,Klingon,4,3 +RESPONSE_244,5.331864429829996,-0.10191396433716902,-0.201519083549608,-0.2766909564127052,-0.1071138541761503,4.307419883308694,1,-7002.343822486522,436,QUESTION_3,Klingon,4,3 +RESPONSE_245,3.1751635979005672,-0.08660058870261193,-0.4100422439046217,-0.6293019740034831,-0.1453326355725099,4.468266545601916,1,-11664.944355457012,48,QUESTION_3,Esperanto,1,1 +RESPONSE_246,5.087493230754122,-0.08094123823576686,-0.21307026139830243,-0.27798630657761875,-0.11435146816048652,4.040235282133917,20,-9161.728286462201,308,QUESTION_3,Vulcan,3,3 +RESPONSE_247,4.469437625560663,-0.06112563830021224,-0.18012543925919952,-0.5083579218050028,-0.12508822329078162,4.190682712144235,10,-12216.09317341963,186,QUESTION_3,Esperanto,3,3 +RESPONSE_248,5.153930582216271,-0.11884064381707285,-0.223003610358311,-0.28765181306726106,-0.11929034142322636,4.326970780506847,6,-9013.382107838052,349,QUESTION_3,Klingon,4,4 +RESPONSE_249,5.884141407814321,-0.05841034627573113,-0.1060355397109696,0.0008250790151386926,-0.07545949153047665,4.5578240641279395,5,-11982.3531761978,357,QUESTION_3,Esperanto,5,4 +RESPONSE_250,4.937824503802121,-0.10652055594519418,-0.233914304582324,0.000861599745771921,-0.13262160801178852,4.617238351885399,9,-11622.2603358142,285,QUESTION_3,Klingon,3,3 +RESPONSE_251,4.986010214583632,-0.0851076130159613,-0.2329626538812769,-0.18763836404216383,-0.12621123772592543,4.187181037582449,17,-5185.155882112639,307,QUESTION_3,Navi,3,2 +RESPONSE_252,4.786593205811598,-0.08233241850947934,-0.13806158403386595,-0.1595169225687501,0.0003143041221849308,4.505049741000845,18,-10865.965237738465,357,QUESTION_3,Esperanto,3,3 +RESPONSE_253,5.743005956030305,-0.08056250594373551,-0.077392376954295,0.00126189477102197,-0.05431273536354468,4.612484993675554,10,-11869.19085176186,652,QUESTION_3,Klingon,5,4 +RESPONSE_254,5.415672261649682,-0.13417778474419212,-0.19024355095646742,0.00013108277980199362,-0.09508404382495085,4.378246448080558,7,-9362.693178294403,436,QUESTION_3,Klingon,4,4 +RESPONSE_255,5.517970694463395,-0.12018581755342238,-0.24486658155149993,-0.2529661690511857,0.0014416143096777727,4.836128851097915,19,-12812.603198923594,253,QUESTION_3,Klingon,3,3 +RESPONSE_256,5.29252323771578,-0.08608611127873897,-0.1415125052280382,-0.0008293564277238506,-0.06681885358146891,4.822970004312149,20,-9264.814154275757,447,QUESTION_3,Navi,4,4 +RESPONSE_257,4.737139609794021,-0.08934635287738384,-0.11698178043918457,-0.1514080381769285,-0.09625304995423596,4.984081205188354,19,-11193.020048350083,215,QUESTION_3,Esperanto,3,3 +RESPONSE_258,5.5462832798130295,-0.06274375458880523,-0.13011341001890522,0.00013814418979659474,-0.06616959579675763,4.472362272230946,11,-13089.736503074575,472,QUESTION_3,Navi,4,4 +RESPONSE_259,5.200130591950383,-0.10576069326721327,-0.16380480556997135,-0.4552522078906043,-0.15367570192830146,4.024176868687386,3,-6647.052607382991,378,QUESTION_3,Klingon,3,3 +RESPONSE_260,5.1555745186953805,-0.09652472649531334,-0.1332133332487251,-0.32963913106348974,-0.09619409726047409,4.548004168286978,15,-13073.486200228715,330,QUESTION_3,Esperanto,4,4 +RESPONSE_261,5.217726653139803,-0.1141204515769663,-0.21053860117206605,-0.3069179868749838,-0.15456614015171455,4.539130472082162,9,-6252.141138665949,333,QUESTION_3,Navi,3,4 +RESPONSE_262,5.32096874736184,-0.09780866797288243,-0.23411069970142503,-0.26611401777495675,-0.049535491714057536,4.113224761902354,14,-8930.33210534653,411,QUESTION_3,Vulcan,3,3 +RESPONSE_263,6.621917167022833,-0.08788603666146977,-0.09416424497819004,-0.12729286165376327,-0.07537607962384896,4.556233802514267,4,-8564.24064949547,678,QUESTION_3,Vulcan,6,6 +RESPONSE_264,5.967073913863434,-0.08141364158229644,-0.06283057960892187,0.0014837335688878855,-0.0623701146831819,4.688415237231123,13,-4992.480472343104,778,QUESTION_3,Vulcan,4,4 +RESPONSE_265,4.015404493421033,-0.12159034201183383,-0.2859388050863119,8.479316510188071e-05,-0.09459056448284178,5.075545808903742,13,-12274.956834112958,109,QUESTION_3,Vulcan,3,2 +RESPONSE_266,5.7915601910575605,-0.08732754469147655,-0.26282106578129405,-0.3092207242962449,-0.10538124441562317,4.417463665289311,13,-6250.060496326118,365,QUESTION_3,Esperanto,4,4 +RESPONSE_267,5.152528151857857,-0.10090296692498546,-0.10725681621896889,-0.0012939769883854434,-0.0750380934407703,4.681368240592403,18,-4038.02345721357,352,QUESTION_3,Vulcan,4,4 +RESPONSE_268,5.707916940061986,-0.07198574585877052,-0.07744299800681478,0.0021233446269846618,-0.08817723367311732,4.313883137335946,19,-3464.987113392728,660,QUESTION_3,Esperanto,3,4 +RESPONSE_269,5.841380666197532,-0.07264726260029791,-0.1296710321907877,-0.20036702158185024,-0.10325963078951032,4.499392582014857,11,-9543.590777695355,661,QUESTION_3,Navi,4,4 +RESPONSE_270,4.948916020597414,-0.10206664344053916,-0.2904945384490198,-0.00022840685220212124,-0.09582106326598182,4.3226035906035944,12,-9361.561101248395,437,QUESTION_3,Navi,4,3 +RESPONSE_271,4.315725582139067,-0.09564619187405322,-0.20656123141294136,-0.41526129534484874,-0.18462616890243896,4.674565687966116,8,-6359.374361087734,142,QUESTION_3,Navi,2,2 +RESPONSE_272,5.121892038991521,-0.141145332208798,-0.18417686095792615,-0.30541818111125807,-0.10927409077333693,4.716128548902902,18,-7269.245989716746,331,QUESTION_3,Vulcan,4,4 +RESPONSE_273,5.109629380669668,-0.1322425709884759,-0.23629005817341292,-0.3724542352804065,-0.07720416355002309,4.256034373877909,18,-5295.219149916035,330,QUESTION_3,Klingon,3,4 +RESPONSE_274,4.85048179291434,-0.1378710102708129,-0.2971516228541129,-0.5130476377824987,-0.17771947374319677,4.8165763878582535,9,-6334.7846543327605,124,QUESTION_3,Navi,2,2 +RESPONSE_275,5.0619234403498705,-0.09335350925132857,-0.19168841893672733,-0.3243897356886625,-0.1270236229784811,4.003646593544001,13,-9472.44861752283,312,QUESTION_3,Vulcan,3,3 +RESPONSE_276,5.259276736451545,-0.06772899007071259,-0.25104346013974704,-0.00012994806117608553,-0.08791051704775098,4.543663662249629,7,-10489.22402922359,393,QUESTION_3,Klingon,4,4 +RESPONSE_277,6.002127456847859,-0.07613658134766157,-0.0636430004892078,-0.05140004298063137,-0.09045276057433309,4.736457990979722,14,-8168.692883356046,736,QUESTION_3,Klingon,4,5 +RESPONSE_278,5.3153403627390645,-0.12981371403309538,-0.1859543394308223,-0.22308405989754887,-0.15866792686989015,4.573855664357426,14,-10192.359238718274,398,QUESTION_3,Klingon,3,4 +RESPONSE_279,4.831775459586885,-0.08635936842310199,-0.1728706269748487,-0.27036484908607133,-0.10623684076689789,4.4923986979218125,14,-7491.874265381262,266,QUESTION_3,Navi,4,4 +RESPONSE_280,4.643564007070344,-0.12537189280724723,-0.05615873822289266,-0.32421135032331894,-0.08022285192896521,4.5695856300372775,12,-7433.107011085357,314,QUESTION_3,Navi,4,4 +RESPONSE_281,3.9738832021111774,-0.2497784583147999,-0.2355721248733889,-0.533166651758528,-0.1951925975152189,3.288145670141757,6,-10096.6344260269,54,QUESTION_3,Vulcan,1,1 +RESPONSE_282,5.288165556042248,-0.07278756561582131,-0.09160008218272424,-0.16754641204902251,-0.05188507278861851,4.637019619977687,3,-8215.739661474538,370,QUESTION_3,Navi,4,4 +RESPONSE_283,5.160000496238821,-0.08374200564739054,-0.18924341333291997,-0.17631525305875037,-0.12804770320819453,4.355759056264475,14,-12497.212282424327,361,QUESTION_3,Navi,4,4 +RESPONSE_284,5.361923561522162,-0.10521746042163299,-0.20212791853511922,0.0008135806052107496,-0.09551606434357282,4.2091062395639,1,-6420.7142930327545,449,QUESTION_3,Esperanto,4,4 +RESPONSE_285,5.146287184845587,-0.094148946557656,-0.17351032275302067,-0.05232666600618985,-0.07179215926461775,4.594882236980965,6,-12887.574568107704,369,QUESTION_3,Esperanto,4,4 +RESPONSE_286,6.307139323402259,-0.08998299824486036,-0.1507000564478421,-0.2738923430729638,-0.04459920430580796,4.397089250352048,10,-9196.514847238153,526,QUESTION_3,Esperanto,4,4 +RESPONSE_287,5.332562083709864,-0.07225254213064615,-0.21131307519401504,-0.20405593148404455,-0.0896123435008248,4.3370752339894105,11,-9816.803041317238,378,QUESTION_3,Klingon,3,3 +RESPONSE_288,6.074510429547948,-0.11289270730568111,-0.1509242307290253,-0.09417676731236573,-0.09450668330130149,4.656119516754376,18,-4919.562973378247,877,QUESTION_3,Navi,5,6 +RESPONSE_289,5.039949941181753,-0.07220550891015415,-0.2370343375955098,-0.1754085355251447,-0.054979209789505386,4.477342380228099,14,-9833.554221099475,321,QUESTION_3,Vulcan,3,3 +RESPONSE_290,5.290031658203198,-0.12308012980979571,-0.2253269642487426,-0.3692754832512613,-0.15012445022287618,4.439286101118827,12,-6337.388106869267,181,QUESTION_3,Esperanto,3,3 +RESPONSE_291,5.5432240880382135,-0.10552698433215175,-0.06167222744214342,0.0007584894631370701,-0.061239078884366235,4.670365332119746,3,-11133.888979519481,540,QUESTION_3,Klingon,4,4 +RESPONSE_292,5.376848328634963,-0.10332761186163071,-0.12213518079125292,-0.2817019418256919,-0.0703869781100991,4.444546940549332,2,-5291.7844534344495,401,QUESTION_3,Klingon,4,4 +RESPONSE_293,5.432640275835587,-0.12194078999109927,-0.18807390376358502,-0.2833125925452306,-0.1158041167960683,4.152839293369808,10,-5256.355051603282,220,QUESTION_3,Klingon,2,3 +RESPONSE_294,5.673300477873887,-0.06735095932706454,-0.09917083186431995,-0.25346202783001404,-0.09056314892948016,4.276086978951575,2,-11746.580456632484,605,QUESTION_3,Esperanto,4,4 +RESPONSE_295,4.530904139731825,-0.13608288151379208,-0.22477564027219138,-0.36952846472133577,-0.1520670232765978,4.0646577035229035,15,-5637.231531144967,178,QUESTION_3,Esperanto,1,2 +RESPONSE_296,5.277061967746415,-0.11902768040711192,-0.3302901947437292,-0.3169735100322647,-0.1570258062082516,3.9164987505799935,2,-7623.376978073809,371,QUESTION_3,Esperanto,3,3 +RESPONSE_297,5.134426466570907,-0.0934420855224018,-0.18679911258683848,-0.28889065418909354,-0.12367273024423445,4.569983379328131,4,-8616.301792129114,320,QUESTION_3,Navi,3,3 +RESPONSE_298,5.083751296506639,-0.10675131044363151,-0.2648786298642519,-0.36062916995619626,-0.0750919359840711,3.823843954568697,2,-3576.867305402503,355,QUESTION_3,Esperanto,4,4 +RESPONSE_299,5.571734084842831,-0.0939603620074604,-0.07571811952198093,-0.22562394518201404,-0.09753798990622672,3.997505315662353,10,-9925.35837843199,530,QUESTION_3,Esperanto,3,3 +RESPONSE_300,4.584991801686085,-0.11381788961885815,-0.20030274602299647,-0.20198472049550925,-0.1010294255291337,4.871639936111704,15,-10192.827818655533,198,QUESTION_3,Klingon,4,3 +RESPONSE_301,4.955305091867469,-0.05781490175674248,-0.08518383315217656,-0.31561933618810334,-0.060794394763901784,4.727533322745031,18,-8157.446654442529,279,QUESTION_4,Klingon,3,3 +RESPONSE_302,5.399713143028155,-0.07864143646901009,-0.15952072445869173,-0.27152093248901477,-0.10661865516977966,4.821487835739876,17,-5155.629436274153,434,QUESTION_4,Esperanto,4,4 +RESPONSE_303,4.880701444296479,-0.05947429189915037,-0.13911487641204925,-0.30086783599880546,0.0016349662367300827,4.6936199093998585,13,-3898.4709603698802,261,QUESTION_4,Klingon,3,3 +RESPONSE_304,4.737735466531022,-0.05289532664273119,-0.20853275855070608,-0.3142125998254392,-0.06623876261939347,4.421630182181217,19,-10631.818402336847,224,QUESTION_4,Vulcan,3,3 +RESPONSE_305,4.692318175635321,-0.11666912582067768,-0.12096578897941089,-0.261910269753735,-0.1201278124939143,5.06742165915298,5,-5700.994408098668,204,QUESTION_4,Navi,3,3 +RESPONSE_306,5.4899120842677975,-0.06197910387323672,-0.2086488164583563,0.0006691744402271521,-0.10132549934592532,4.564234952155773,17,-10433.097580660726,504,QUESTION_4,Esperanto,4,4 +RESPONSE_307,5.041767462984724,-0.07212765882689066,-0.11345111004224813,-0.17676716400261053,-0.055639262142878364,4.583097744895763,10,-11975.915167423456,314,QUESTION_4,Navi,3,3 +RESPONSE_308,4.757676176366988,-0.10699452040851781,-0.203628223661115,-0.0662999015771136,-0.14613601097681592,4.97475053442119,5,-10381.28400705906,237,QUESTION_4,Klingon,3,4 +RESPONSE_309,5.487339636854778,-0.12038970066593535,-0.2649183876387064,-0.22351632973193666,-0.11349361761167315,3.8963021311648744,20,-8827.078964060018,474,QUESTION_4,Navi,3,3 +RESPONSE_310,6.31234843837154,-0.11161858891948064,-0.32705975319786273,-6.712392649389326e-05,-0.059854466199563194,4.344633282024866,10,-6526.148934994872,528,QUESTION_4,Vulcan,2,2 +RESPONSE_311,6.096468546762044,-0.0814340906872277,-0.14612677253363046,0.0005619550026544653,-0.12081710534301032,4.235983392174054,10,-6763.569852626411,469,QUESTION_4,Vulcan,3,3 +RESPONSE_312,5.366214397769209,-0.07748914605037481,-0.18933953693521013,0.00149679163155383,-0.04999764722463351,4.590054780051795,10,-5764.183685097144,397,QUESTION_4,Vulcan,4,4 +RESPONSE_313,5.140276248810937,-0.06949900535456154,-0.1977200304405104,-0.15690410516883202,-0.07332759072830472,4.466277673198027,11,-11282.256887522497,356,QUESTION_4,Navi,3,3 +RESPONSE_314,5.49720684780614,-0.11673254525918592,-0.06359038851473617,0.0006138960734023117,-0.09150816365786206,4.5380244022240115,6,-7725.018465020535,484,QUESTION_4,Vulcan,4,4 +RESPONSE_315,4.388967064001772,-0.15947359470041855,-0.396767260149171,-0.3494079268790149,-0.07634656758240493,4.825722717000839,18,-4979.849195864123,170,QUESTION_4,Klingon,2,2 +RESPONSE_316,5.32945064872067,-0.11587489824901069,-0.15189092674702045,-0.1306928562100161,-0.06962743010465337,4.050962984837867,20,-5204.32383545314,405,QUESTION_4,Klingon,3,4 +RESPONSE_317,6.107908598846432,-0.07551043108692923,-0.2763064493856387,-0.0013357831009144132,-0.16286528702220598,3.6552046169951726,3,-9968.596141792006,493,QUESTION_4,Klingon,3,3 +RESPONSE_318,5.666187276220895,-0.06748789590901787,-0.07038806007036333,-0.17083539899948777,-0.00032602981269758607,4.775136807613845,18,-6434.19092457932,593,QUESTION_4,Esperanto,4,5 +RESPONSE_319,4.537763542187966,-0.05770236714185613,-0.19612660186039893,-0.3111683333187032,0.0002461805020160936,4.26093261664816,11,-8258.54869763328,185,QUESTION_4,Vulcan,3,3 +RESPONSE_320,4.788853836830869,-0.15460610473381997,-0.3077564631326269,-0.18723318983095755,-0.13038790033281117,4.519930879627684,9,-5991.673100673665,232,QUESTION_4,Navi,3,3 +RESPONSE_321,4.607383885546701,-0.09605119859473947,-0.19013984846405702,0.000294952158264376,0.00011031065755837562,4.523935458614097,14,-9760.166907890956,196,QUESTION_4,Esperanto,3,3 +RESPONSE_322,4.744424094730305,-0.12198213989976885,-0.1900968548288138,-0.3732261774682136,-0.09088969850019926,4.2534173441929575,13,-12145.31586066551,238,QUESTION_4,Klingon,3,3 +RESPONSE_323,5.463273180273001,-0.11571011221990395,-0.19191146414185115,-0.19153967657966284,-0.154043394597514,4.524167206686124,16,-9785.728828669224,250,QUESTION_4,Esperanto,3,3 +RESPONSE_324,5.427955238265926,-0.10584767139062853,-0.10832197945826834,-0.06133472148366697,-0.0760577208118173,4.475130417529484,3,-11756.770713709137,521,QUESTION_4,Esperanto,4,4 +RESPONSE_325,5.693091382393225,-0.11166176534640188,-0.10254006209611992,-0.08300784403099692,-0.09213569352240793,4.258241243089646,18,-9597.733907355514,570,QUESTION_4,Navi,4,4 +RESPONSE_326,5.01319977087655,-0.07259555652523235,-0.15465060707166456,-0.24502494693800939,-0.0822621383318265,4.487963478842092,19,-10273.501728541962,292,QUESTION_4,Navi,3,4 +RESPONSE_327,5.55636819686542,-0.082833214246711,-0.09980150402359195,-0.2735148858770635,-0.0756467980661404,4.676839087795182,11,-12803.340804298123,510,QUESTION_4,Klingon,4,4 +RESPONSE_328,5.945713704537371,-0.08954307344064595,-0.17140245902150128,-0.23071788192792253,-0.10143066715446074,4.276543077302388,9,-7188.378471586086,384,QUESTION_4,Klingon,4,4 +RESPONSE_329,5.306931997826618,-0.0942391709596926,-0.19076307177337826,0.0016100243433471038,-0.10122846396880465,4.442118507999869,4,-4842.148280071611,389,QUESTION_4,Esperanto,4,4 +RESPONSE_330,5.651989961990758,-0.09456196935387672,-0.08174650793291378,0.0006576004467528831,-0.05760726983543669,4.606545813230723,4,-5218.9305960312595,599,QUESTION_4,Esperanto,4,4 +RESPONSE_331,5.094620704027824,-0.12987015280168723,-0.2421714457116207,-0.205483501519745,-0.14262556900122358,4.518411146698394,15,-5236.970524968138,330,QUESTION_4,Klingon,4,3 +RESPONSE_332,5.485200433125546,-0.06716308505699153,-0.13097275249692264,0.00029015301297351676,-0.07861941174832422,4.593074809240415,2,-5693.96909192208,480,QUESTION_4,Klingon,4,4 +RESPONSE_333,4.75204933358063,-0.14711006839325796,-0.31861677656055304,-0.25416614235055246,-0.0658827417353054,4.39573301746834,1,-8770.66546390419,218,QUESTION_4,Esperanto,3,3 +RESPONSE_334,5.326908722502998,-0.10910136234182105,-0.23163850622051507,-0.2005217541624492,-0.11936318776023903,4.3354725987977725,14,-11203.70947969342,428,QUESTION_4,Klingon,3,3 +RESPONSE_335,5.25978723440872,-0.10632278923297371,-0.29602670397769254,-0.3189286195630757,-0.16475852890520498,3.8684580551271504,2,-7957.696509462288,366,QUESTION_4,Esperanto,4,4 +RESPONSE_336,5.690670144841193,-0.1830958707592406,-0.24772573399798484,-0.42283890849351097,-0.11893753030979994,4.233294326546961,17,-11394.576955362481,276,QUESTION_4,Esperanto,3,3 +RESPONSE_337,4.430752859974619,-0.09428452775440632,-0.22179130929636448,-0.2632827016061192,-0.14661175574572088,5.025055126002049,9,-8311.477789986486,177,QUESTION_4,Vulcan,3,3 +RESPONSE_338,5.249218319902717,-0.11424583764858783,-0.09325979254907525,-0.11780192551440796,-0.10225384984248675,4.558870162256449,20,-10308.832237045168,584,QUESTION_4,Esperanto,4,4 +RESPONSE_339,5.301550929597822,-0.09449054062456162,-0.10233320308643573,-0.0003059545814060173,-0.14257207995202512,4.642884562102016,5,-4116.1535336637,384,QUESTION_4,Vulcan,3,3 +RESPONSE_340,5.533138259738684,-0.10337225761174006,-0.07874567014253582,-0.10202176402282591,-0.06444079093497886,4.621161615880386,12,-5131.776991393341,485,QUESTION_4,Navi,5,4 +RESPONSE_341,5.09376879256786,-0.06973865985730417,-0.19940104630838565,0.0005939625512907878,-0.09451570360778981,4.358447805509245,19,-3990.868173720122,451,QUESTION_4,Esperanto,3,3 +RESPONSE_342,5.047017767741775,-0.0952794882102766,-0.16972314643088085,-0.37308427861562377,-0.12736691007623555,4.612642575582045,18,-5937.45312353181,309,QUESTION_4,Vulcan,3,4 +RESPONSE_343,4.989597731190521,-0.08127299432672515,-0.25208599902646556,0.0007668743113992809,-0.1351325770640105,4.266216692603613,9,-6889.8520389130945,280,QUESTION_4,Vulcan,4,3 +RESPONSE_344,5.376583520705638,-0.10356254580531483,-0.20735794201697136,-0.2769811840178405,-0.10736138166983199,4.3759645262365225,12,-5651.129671811191,436,QUESTION_4,Vulcan,4,3 +RESPONSE_345,3.210276595036554,-0.08601810919445936,-0.41160366750408256,-0.6282106136730936,-0.14495067170572248,4.540561427284394,10,-6021.310736890266,48,QUESTION_4,Navi,1,1 +RESPONSE_346,5.040841377850831,-0.08046083012171328,-0.21347093037677262,-0.2807099474675898,-0.11429504306156474,4.050054927960239,15,-10628.562848211486,308,QUESTION_4,Klingon,3,3 +RESPONSE_347,4.618979624146532,-0.06252705668158734,-0.17863098188739468,-0.5080153316195212,-0.12573746309058553,4.155734774528175,5,-10577.296334401588,186,QUESTION_4,Klingon,3,3 +RESPONSE_348,5.13661184337689,-0.11992354763142647,-0.22103516303840245,-0.2897388475908073,-0.12062512089438591,4.254221746263506,6,-8181.292183858089,349,QUESTION_4,Esperanto,4,4 +RESPONSE_349,5.940947270114374,-0.05789024532392305,-0.10427536368444527,-0.0016293720194083805,-0.0747000089506362,4.732211576452984,18,-8121.2280247162735,357,QUESTION_4,Navi,5,4 +RESPONSE_350,4.9912079925308985,-0.10617142102386337,-0.23668218108331462,-0.0005960381403031904,-0.13332244301642773,4.57162098955175,5,-10219.21313287398,285,QUESTION_4,Esperanto,3,3 +RESPONSE_351,5.088570187460814,-0.08437645035260094,-0.23581670519555586,-0.19028431502438284,-0.1272878377391841,4.147984552162167,2,-9686.069403605297,307,QUESTION_4,Vulcan,3,2 +RESPONSE_352,4.788784999060399,-0.08154014096885352,-0.1385089410702792,-0.15706635185004147,-0.0003539761095220311,4.546046315571217,16,-5180.903133253684,357,QUESTION_4,Esperanto,3,3 +RESPONSE_353,5.805662224738019,-0.08055874027923692,-0.07920663442128013,-0.0007119090324199066,-0.05546327453750173,4.578761265046937,14,-12236.460097263918,652,QUESTION_4,Navi,5,4 +RESPONSE_354,5.3779909716648495,-0.1319147931181761,-0.18986746441365712,0.00043322176758411467,-0.09423438642059559,4.4378585665929755,6,-7430.437207180634,436,QUESTION_4,Vulcan,4,4 +RESPONSE_355,5.533017011016815,-0.11950029643100753,-0.24225862988682162,-0.2534945547243088,-0.0010538931197913166,4.8127069259202395,17,-6631.665482479998,253,QUESTION_4,Klingon,3,3 +RESPONSE_356,5.35725653887001,-0.08549831747749252,-0.14257928029255462,0.00192554931877437,-0.06662136523667483,4.831524553161588,2,-4470.229919208699,447,QUESTION_4,Klingon,4,4 +RESPONSE_357,4.605438304104273,-0.0886503256958623,-0.12099422136062347,-0.15358522247184866,-0.09564466522567885,4.979396512411463,4,-7399.438180921511,215,QUESTION_4,Esperanto,3,3 +RESPONSE_358,5.4820842288751885,-0.061566794762768176,-0.12958662682752775,0.0006572158990309818,-0.06492736316085833,4.532199692808122,4,-5305.431252939701,472,QUESTION_4,Klingon,4,4 +RESPONSE_359,5.127261388680495,-0.10511776315033991,-0.16437171913654486,-0.4538662675773652,-0.15395748565003806,4.05672384009018,1,-4833.9977066561305,378,QUESTION_4,Klingon,3,3 +RESPONSE_360,4.992736676529479,-0.09623415526189179,-0.1332163654333108,-0.33257208341618566,-0.09645606881524052,4.626740711816868,7,-6829.010736731042,330,QUESTION_4,Navi,4,4 +RESPONSE_361,5.0121384808556595,-0.11490701765755748,-0.21068526868500806,-0.302072498769989,-0.15514161460721299,4.485643128466942,11,-8998.39153535288,333,QUESTION_4,Navi,3,4 +RESPONSE_362,5.401971554729816,-0.09670933394508677,-0.23223829209496463,-0.26884935095408635,-0.049582444031086775,4.1401700958653675,14,-6199.920652464655,411,QUESTION_4,Klingon,3,3 +RESPONSE_363,6.4971403777668195,-0.08864309635306898,-0.09368811998118834,-0.1269232787935868,-0.07679344501167146,4.520454077475788,5,-9426.413941950046,678,QUESTION_4,Vulcan,6,6 +RESPONSE_364,5.944907534759253,-0.08092223848544067,-0.061703281664682495,1.5773289260129117e-05,-0.06196931517570431,4.627126364780412,16,-11122.885872311972,778,QUESTION_4,Klingon,4,4 +RESPONSE_365,3.9952305723599926,-0.12094032761005713,-0.28981779531702007,0.001887522855449391,-0.09623165825126567,5.0607183242746885,11,-4419.711761559004,109,QUESTION_4,Navi,3,2 +RESPONSE_366,5.998072413465292,-0.08658990039113171,-0.26523112226994494,-0.30841924435182944,-0.10349613345993777,4.364437240358259,13,-9168.87633527167,365,QUESTION_4,Vulcan,4,4 +RESPONSE_367,5.143026635453723,-0.10111877832557402,-0.10568755313175522,0.0016974899259111175,-0.0760774053457474,4.658043926161678,10,-11212.592392914774,352,QUESTION_4,Klingon,4,4 +RESPONSE_368,5.893195776429378,-0.07164555067063207,-0.0789025779983802,-0.0020703078300907084,-0.08771457524031644,4.333038107742827,19,-11035.010304057692,660,QUESTION_4,Navi,3,4 +RESPONSE_369,5.836500785417224,-0.07274838577809443,-0.12939373104507335,-0.19939648690569253,-0.10164237403430046,4.562409247475463,10,-4471.610894327133,661,QUESTION_4,Navi,4,4 +RESPONSE_370,5.060345884119955,-0.10028061896599003,-0.2895992875054389,0.00045439525565571786,-0.09523012724982122,4.281353080679009,11,-10775.892149119542,437,QUESTION_4,Vulcan,4,3 +RESPONSE_371,4.270945346516098,-0.09829582702967823,-0.2100305905678328,-0.41287704246355245,-0.1878075978253375,4.633066930852437,12,-6661.416651288943,142,QUESTION_4,Esperanto,2,2 +RESPONSE_372,5.05344196588253,-0.14489303811171392,-0.1840916674647714,-0.3050762944445362,-0.10825384933027223,4.638555438321398,2,-4768.3048290696,331,QUESTION_4,Esperanto,4,4 +RESPONSE_373,5.153003386250752,-0.13332504511554036,-0.23536483154583945,-0.3710050062513259,-0.07778207588580355,4.264812178356471,18,-7377.478603021145,330,QUESTION_4,Klingon,3,4 +RESPONSE_374,4.752140944038631,-0.13965196899197307,-0.2940852857484721,-0.5112710547783036,-0.17554803371798167,4.796865327182454,10,-9288.156440945237,124,QUESTION_4,Vulcan,2,2 +RESPONSE_375,5.0877265866837265,-0.09382384509244833,-0.19545336583590286,-0.32728392775425147,-0.1279546644913714,4.092331631112595,12,-12440.052591045811,312,QUESTION_4,Klingon,3,3 +RESPONSE_376,5.388702707834113,-0.06745706252636954,-0.25061110310291707,-0.0008979535956198197,-0.08762361635701002,4.599632772763745,20,-7398.592669126122,393,QUESTION_4,Esperanto,4,4 +RESPONSE_377,5.821055707351379,-0.07561801968791057,-0.06339482533834527,-0.05122932109716607,-0.08994523156427124,4.662969013236259,13,-6397.290568541508,736,QUESTION_4,Esperanto,4,5 +RESPONSE_378,5.256482070876538,-0.13039756744128347,-0.1892384291917802,-0.22019880345134193,-0.16043339568047454,4.596753886256454,17,-7171.513135021626,398,QUESTION_4,Klingon,3,4 +RESPONSE_379,4.885057298969092,-0.08600643913044696,-0.1748857347730133,-0.2685156306050418,-0.10606973375030747,4.402676188031244,9,-12821.245692443104,266,QUESTION_4,Navi,4,4 +RESPONSE_380,4.6074001621489264,-0.12788705405492767,-0.056114389007438176,-0.31471449355438075,-0.07884426835509409,4.51088576835952,13,-4613.200611783748,314,QUESTION_4,Klingon,4,4 +RESPONSE_381,4.053658772720239,-0.2464324684253107,-0.23488934365648723,-0.522622663705403,-0.19030675402773456,3.2288176225862797,16,-4861.367183322626,54,QUESTION_4,Vulcan,1,1 +RESPONSE_382,5.168097287847253,-0.07334451536525449,-0.08917539417286663,-0.16377370662598556,-0.05240383818103121,4.647596871855687,5,-13421.860321058079,370,QUESTION_4,Esperanto,4,4 +RESPONSE_383,5.257751488713167,-0.08370146836892667,-0.19181833023239453,-0.17525269785719977,-0.12859400750926508,4.313456848964917,13,-11298.663089710355,361,QUESTION_4,Vulcan,4,4 +RESPONSE_384,5.42401365686827,-0.10547412290672942,-0.20172790454825348,-0.00015171197778683437,-0.09559299097633073,4.178114223179643,14,-11919.623273769683,449,QUESTION_4,Klingon,4,4 +RESPONSE_385,5.233756752096073,-0.09782095751429315,-0.17084012692671055,-0.05176070230190802,-0.07272489399285582,4.53610602152856,17,-8626.447900290363,369,QUESTION_4,Vulcan,4,4 +RESPONSE_386,6.23624520998955,-0.08833484272887095,-0.1488784313724717,-0.2691066600850128,-0.04331366835202882,4.419379356691029,16,-6599.7505770792595,526,QUESTION_4,Klingon,4,4 +RESPONSE_387,5.204436637796426,-0.07104361363124607,-0.20841301453743682,-0.20633177617996395,-0.08882807402104397,4.293496745983118,5,-5617.7767993442085,378,QUESTION_4,Vulcan,3,3 +RESPONSE_388,6.041520928160548,-0.1117299570408031,-0.14563329200189426,-0.09553978458686874,-0.09521864264977115,4.7873127658296015,15,-8752.576102507564,877,QUESTION_4,Klingon,5,6 +RESPONSE_389,5.042887740611892,-0.07045143169022874,-0.24141786536698379,-0.17745253241976083,-0.05648729824890799,4.457156750962269,9,-7631.7426893228,321,QUESTION_4,Vulcan,3,3 +RESPONSE_390,5.19105006831107,-0.12311089401823555,-0.2232725270627964,-0.372161540431348,-0.1490429106037301,4.357657926058178,18,-5607.444667016813,181,QUESTION_4,Esperanto,3,3 +RESPONSE_391,5.477926426413459,-0.10362178014182817,-0.05953280514082254,-0.001458338824553177,-0.060246046184577975,4.688115871425387,20,-8555.110499111901,540,QUESTION_4,Esperanto,4,4 +RESPONSE_392,5.276322494182682,-0.10036435223594725,-0.12316023845296407,-0.2776147112880428,-0.07053972397123623,4.337337868880987,7,-8747.16850280436,401,QUESTION_4,Klingon,4,4 +RESPONSE_393,5.43161087959201,-0.12100097618031862,-0.19129483491667268,-0.2885486704989048,-0.11622935263014608,4.254769168409341,20,-5060.564250563348,220,QUESTION_4,Vulcan,2,3 +RESPONSE_394,5.765194581020047,-0.06772281009703204,-0.10002765319951784,-0.2556449587820199,-0.08931346773757477,4.4026633746090935,19,-10763.723420628354,605,QUESTION_4,Vulcan,4,4 +RESPONSE_395,4.539413191546669,-0.13924939889486326,-0.22587193087204663,-0.3626335577327069,-0.1503261751447313,4.093366539886849,11,-6931.918938544225,178,QUESTION_4,Esperanto,1,2 +RESPONSE_396,5.298055696281496,-0.1184453009749855,-0.33162593007400387,-0.32297447359666337,-0.15561667230366658,3.9311727467573383,18,-11086.69553574274,371,QUESTION_4,Esperanto,3,3 +RESPONSE_397,5.144885575630544,-0.09252383423854943,-0.18265424496654786,-0.28657505479740514,-0.12528072093980286,4.625057447937676,4,-4405.903826255953,320,QUESTION_4,Esperanto,3,3 +RESPONSE_398,5.149804717943142,-0.10542036920259916,-0.26748991983450904,-0.35949228103912134,-0.07520043264085383,3.894206003366409,10,-5983.2861127057995,355,QUESTION_4,Esperanto,4,4 +RESPONSE_399,5.620002244027703,-0.09292009816887108,-0.07442308483515415,-0.22566679359966038,-0.09720622132234215,3.980790590109307,18,-5934.236350926868,530,QUESTION_4,Esperanto,3,3 +RESPONSE_400,4.550479416763726,-0.11454647069741722,-0.20111964073825864,-0.2031291442576167,-0.09979972530898554,4.762700699796302,18,-10131.73522019506,198,QUESTION_4,Klingon,4,3 +RESPONSE_401,4.915081590595693,-0.05813466719968557,-0.08522463806885991,-0.3218340118411269,-0.060571807237155974,4.746650127813602,19,-11042.826470358637,279,QUESTION_5,Esperanto,3,3 +RESPONSE_402,5.394000247727426,-0.07676943558221014,-0.15925643843725643,-0.2733075133462907,-0.10677254789703224,4.775308615270717,9,-4995.085462664544,434,QUESTION_5,Klingon,4,4 +RESPONSE_403,4.8869588318090695,-0.059713182228368106,-0.13848912076314704,-0.30185521285135775,0.0007584928297359549,4.681534193082517,1,-8197.14086125997,261,QUESTION_5,Klingon,3,3 +RESPONSE_404,4.708296133418728,-0.05351187688988709,-0.21222803092909895,-0.3027139000588446,-0.06622421696912384,4.41810394262162,16,-6749.106810192595,224,QUESTION_5,Vulcan,3,3 +RESPONSE_405,4.627057409170962,-0.11795819319724746,-0.12227439915854264,-0.25752675938692215,-0.12036308997198261,5.0631171766172995,19,-9983.874034487846,204,QUESTION_5,Klingon,3,3 +RESPONSE_406,5.636479159231367,-0.05998226866099605,-0.20504982949144063,0.0016323031281203816,-0.10009903832404446,4.5457484607367205,9,-4080.1148889957285,504,QUESTION_5,Esperanto,4,4 +RESPONSE_407,5.077780741778928,-0.07177722055652706,-0.11220843227502642,-0.18153799568780515,-0.05686636967196628,4.6410035857723075,12,-11100.283501751095,314,QUESTION_5,Esperanto,3,3 +RESPONSE_408,4.753845898801028,-0.10987063355709474,-0.20313609826221773,-0.06464114168024479,-0.14503034452714458,5.001958555840137,3,-10894.77337504161,237,QUESTION_5,Klingon,3,4 +RESPONSE_409,5.523896472898773,-0.11808440916040397,-0.2570189853425045,-0.22642436082224948,-0.11409212896844201,3.812361227257866,3,-6306.961603873759,474,QUESTION_5,Vulcan,3,3 +RESPONSE_410,6.456597232154774,-0.11243443268192481,-0.32717029685329757,0.00038661550829034236,-0.061596810262927126,4.237194587882097,3,-8930.26439229531,528,QUESTION_5,Esperanto,2,2 +RESPONSE_411,6.043918779874865,-0.08213924564573487,-0.14562515996172368,-4.8111598667129556e-05,-0.12171759807274153,4.149833596355787,11,-6707.667533307922,469,QUESTION_5,Navi,3,3 +RESPONSE_412,5.371115263378051,-0.07797179922718396,-0.18599711204931896,-0.00259835475381056,-0.05059982578249962,4.530484231880471,12,-7706.234709434626,397,QUESTION_5,Esperanto,4,4 +RESPONSE_413,5.145155556050378,-0.06963180491778889,-0.19638322368471872,-0.16038478429309308,-0.07470388775083175,4.401559487949379,11,-7484.585566030518,356,QUESTION_5,Navi,3,3 +RESPONSE_414,5.49849442208692,-0.11471695931595387,-0.0650816012297397,-0.00020915588579281576,-0.0901681026130291,4.564764945844017,12,-11988.043365562346,484,QUESTION_5,Esperanto,4,4 +RESPONSE_415,4.494758033930074,-0.15670720643187838,-0.3857144634576797,-0.35145137829893824,-0.0761727512118645,4.772668039761474,18,-8835.7719305697,170,QUESTION_5,Navi,2,2 +RESPONSE_416,5.298093965935569,-0.1167785034023714,-0.14735929227449093,-0.13087813805020762,-0.06885444161754578,4.053045886365079,6,-13804.557592883206,405,QUESTION_5,Navi,3,4 +RESPONSE_417,6.177006632513792,-0.07534137584306669,-0.2763727634982777,0.0005734281703423849,-0.16404415608843306,3.664261537732086,19,-11661.021859142496,493,QUESTION_5,Klingon,3,3 +RESPONSE_418,5.758483736437616,-0.067244687176744,-0.07047024026347536,-0.16813520962831968,0.0012640439190894522,4.834910487110074,9,-5385.301479955095,593,QUESTION_5,Vulcan,4,5 +RESPONSE_419,4.519629374573698,-0.05840780495723831,-0.19412381446268176,-0.3195439552746211,-0.0007026592891744782,4.161465817197018,4,-12714.608620946397,185,QUESTION_5,Esperanto,3,3 +RESPONSE_420,4.694241838682439,-0.15668233870174184,-0.30307537402398443,-0.1883388155344756,-0.13454228797097711,4.419565033033062,6,-8938.321630020115,232,QUESTION_5,Navi,3,3 +RESPONSE_421,4.579068459362277,-0.0970987526184471,-0.18773151454640943,-0.0003049973457242287,0.00010412664557243396,4.599832747477381,20,-11254.083600299908,196,QUESTION_5,Navi,3,3 +RESPONSE_422,4.754902632872144,-0.12234724452558632,-0.19036927061064884,-0.36985944264726467,-0.09376569440670184,4.3638262050397145,16,-11239.280766207996,238,QUESTION_5,Klingon,3,3 +RESPONSE_423,5.589876954922449,-0.11638954503717314,-0.1902095473263999,-0.18792640047403078,-0.1561927432539656,4.637241861632761,13,-11182.713475948009,250,QUESTION_5,Navi,3,3 +RESPONSE_424,5.57539937155354,-0.10592797844507318,-0.10718582659039519,-0.06113775754868979,-0.07755024032019354,4.492370064512235,5,-11330.27879746294,521,QUESTION_5,Klingon,4,4 +RESPONSE_425,5.607169586652364,-0.11247704760351705,-0.10104817197775805,-0.08375761289899,-0.09257824869061523,4.427559782231795,2,-10865.09888277229,570,QUESTION_5,Vulcan,4,4 +RESPONSE_426,4.996197171104436,-0.07290542951774182,-0.15525552995983843,-0.2503059910560553,-0.08119917526804728,4.456112450342235,13,-12439.686081542184,292,QUESTION_5,Esperanto,3,4 +RESPONSE_427,5.454540661732327,-0.08218800449968688,-0.09980082538322434,-0.27607809075251216,-0.0761151459871873,4.654278154459028,1,-10463.109500113467,510,QUESTION_5,Esperanto,4,4 +RESPONSE_428,6.044833266013086,-0.08921251587587302,-0.16870724640717105,-0.22690183190473806,-0.10316518729292305,4.267558643528476,5,-5899.301701619815,384,QUESTION_5,Klingon,4,4 +RESPONSE_429,5.315228802309022,-0.09485612228104502,-0.18906027828419852,-0.0008138374208533874,-0.10156685941025176,4.4671628457058725,19,-11484.931953960897,389,QUESTION_5,Klingon,4,4 +RESPONSE_430,5.680379892259679,-0.0959255984293298,-0.08039485013689827,0.0010200936307076121,-0.05788857675553898,4.608487883788318,18,-8154.052401633653,599,QUESTION_5,Klingon,4,4 +RESPONSE_431,5.11109039898275,-0.1281604797328038,-0.24229093348707437,-0.20708114209614134,-0.14695872657826106,4.442135587809552,9,-7460.79950586299,330,QUESTION_5,Vulcan,4,3 +RESPONSE_432,5.578237874402767,-0.06793369934474362,-0.12662305829630408,-0.00027389806372914654,-0.08056350971244837,4.62550580004733,13,-12381.375605839423,480,QUESTION_5,Vulcan,4,4 +RESPONSE_433,4.710299260504441,-0.14863929513969312,-0.32025089842093984,-0.24993417882876276,-0.06753693011862953,4.311409996740471,5,-6975.039236270044,218,QUESTION_5,Vulcan,3,3 +RESPONSE_434,5.426669713545245,-0.10777374941398284,-0.23507440230723714,-0.20083084794271697,-0.11886325773960883,4.339562278400583,5,-6191.242802427614,428,QUESTION_5,Vulcan,3,3 +RESPONSE_435,5.217946504969524,-0.10587345436014794,-0.30153651545746263,-0.32252504479229005,-0.1657585200821301,3.8627062355410775,5,-7929.314837159645,366,QUESTION_5,Navi,4,4 +RESPONSE_436,5.687196841685341,-0.1826063760001798,-0.2470360396969753,-0.4184452347454175,-0.12145198949730107,4.258314679039909,19,-7489.720742223548,276,QUESTION_5,Esperanto,3,3 +RESPONSE_437,4.465174808007515,-0.0938022053046422,-0.22384757499203553,-0.2574323295557148,-0.1533766852965661,5.143992434648594,19,-6218.983337705352,177,QUESTION_5,Esperanto,3,3 +RESPONSE_438,5.2374942433946785,-0.11497840364455038,-0.09374770253600886,-0.11840476303134201,-0.10261694427912486,4.582583119162319,4,-8002.45022763561,584,QUESTION_5,Esperanto,4,4 +RESPONSE_439,5.3483414127918545,-0.0946853832920962,-0.10282889316374,0.00019182305635568882,-0.14080024073678696,4.631338477784556,1,-6868.245644886583,384,QUESTION_5,Navi,3,3 +RESPONSE_440,5.459806243154585,-0.1027878716991678,-0.07823288423327271,-0.10196596826650275,-0.06497541856447596,4.521729709432123,1,-8383.00778215925,485,QUESTION_5,Esperanto,5,4 +RESPONSE_441,4.963047739749978,-0.06945376216163374,-0.20029309366276282,0.0006413098354855021,-0.09468390400610083,4.41442373333156,7,-5062.632089571973,451,QUESTION_5,Navi,3,3 +RESPONSE_442,5.012782696819511,-0.0954289040078971,-0.17250893137461845,-0.373797803652766,-0.1278635823787524,4.613387453924231,17,-7258.065624904218,309,QUESTION_5,Klingon,3,4 +RESPONSE_443,4.971993270663383,-0.08284687355477144,-0.2522635983214571,0.00036749141967836477,-0.13255132106260853,4.372707805625929,19,-5461.91797646089,280,QUESTION_5,Klingon,4,3 +RESPONSE_444,5.450675046156775,-0.10351541596100852,-0.20664375277728952,-0.27320673260313266,-0.10714928727818818,4.301989715654287,15,-11165.625600321322,436,QUESTION_5,Vulcan,4,3 +RESPONSE_445,3.2448836842918776,-0.08616484183991825,-0.412356692688007,-0.6220681139791858,-0.14631716894683944,4.508116092855089,10,-5831.998910782598,48,QUESTION_5,Vulcan,1,1 +RESPONSE_446,5.119553207358442,-0.08158500225415852,-0.21315763234713686,-0.2748162313746918,-0.11344845702877074,4.023971639636469,9,-10830.346865140618,308,QUESTION_5,Navi,3,3 +RESPONSE_447,4.561701469231638,-0.06284825428874104,-0.1816841006308739,-0.49852394708127673,-0.12381346973095217,4.23998508091884,10,-9390.33295513956,186,QUESTION_5,Esperanto,3,3 +RESPONSE_448,5.230648360797307,-0.1170876554625286,-0.22091774445611861,-0.2902141915580877,-0.11823681206691311,4.368352127140403,5,-5908.649264467667,349,QUESTION_5,Vulcan,4,4 +RESPONSE_449,5.820887948470274,-0.05896757398295987,-0.10479340665029424,0.0020875633761660837,-0.07520553980453866,4.65531910773623,2,-6865.346291445766,357,QUESTION_5,Esperanto,5,4 +RESPONSE_450,4.919971223074919,-0.1058482357333799,-0.24045102312830066,0.00011713421221327176,-0.1325257841261929,4.566119766073865,20,-7401.265384160716,285,QUESTION_5,Esperanto,3,3 +RESPONSE_451,4.992777677710384,-0.08563806835700408,-0.23487263102821243,-0.19090957092226443,-0.1273293911252986,4.201187765798406,17,-10461.236213652712,307,QUESTION_5,Navi,3,2 +RESPONSE_452,4.771807318078649,-0.07948419362285707,-0.13873018978897847,-0.15864645031582747,0.0008143650656112657,4.527882695765858,5,-13385.395462038694,357,QUESTION_5,Esperanto,3,3 +RESPONSE_453,5.735662325207071,-0.08081871896013983,-0.07834883371685109,0.0012761493207592207,-0.05425739118467959,4.575427836228244,9,-12998.218745676215,652,QUESTION_5,Esperanto,5,4 +RESPONSE_454,5.440448465152179,-0.13163641606586973,-0.18816190842494837,0.0002709685074809869,-0.09532657626521386,4.460324066278157,9,-11977.185242128005,436,QUESTION_5,Navi,4,4 +RESPONSE_455,5.573671816938454,-0.12115274561807553,-0.24674128167921183,-0.24843449107182713,0.0004666985432912171,4.854644545512539,1,-8276.913653607342,253,QUESTION_5,Navi,3,3 +RESPONSE_456,5.2998194940391725,-0.08507850602851655,-0.1389360512811921,-0.0009886306563756085,-0.0672600889481039,4.830174206951339,1,-5894.3970489266285,447,QUESTION_5,Navi,4,4 +RESPONSE_457,4.648148681400997,-0.08827092684631022,-0.11887265779908303,-0.15440987588697377,-0.09704401142042329,5.040469157335116,7,-12464.724322337404,215,QUESTION_5,Klingon,3,3 +RESPONSE_458,5.437132669084892,-0.06145111114727714,-0.12991784754556848,-0.001141823804215901,-0.06559215787310689,4.478789592047987,4,-8838.654955360127,472,QUESTION_5,Esperanto,4,4 +RESPONSE_459,5.290939614924471,-0.10697704762223936,-0.16545301397453482,-0.45204007252337247,-0.15396555021866912,4.033045863702657,13,-8324.518189580516,378,QUESTION_5,Vulcan,3,3 +RESPONSE_460,5.089557550056242,-0.09676684512305343,-0.13531931028706687,-0.3220522291357192,-0.09489966988116859,4.593810794466711,5,-9120.002306566184,330,QUESTION_5,Navi,4,4 +RESPONSE_461,5.009137454539514,-0.11550559946689186,-0.2125437149161621,-0.2996934824158906,-0.15317624303367064,4.547957430384668,1,-10942.395665339913,333,QUESTION_5,Navi,3,4 +RESPONSE_462,5.325139982005796,-0.09708092812667796,-0.23095016974931346,-0.26605275509188125,-0.04945326007349279,4.166267027594351,7,-10332.950115165584,411,QUESTION_5,Klingon,3,3 +RESPONSE_463,6.483882034762735,-0.08885986371677586,-0.09306387298325682,-0.1257735117501317,-0.07613221248597668,4.587722517361377,8,-8591.576746929872,678,QUESTION_5,Esperanto,6,6 +RESPONSE_464,5.969609494191589,-0.08105924470160496,-0.06241799431122988,0.0004807218843402119,-0.06154583819241507,4.635085664296571,6,-5348.824993194257,778,QUESTION_5,Esperanto,4,4 +RESPONSE_465,3.9853957719158486,-0.12253212672847653,-0.2884475911334098,0.000545780550775707,-0.09596819903826871,4.99261128646971,17,-6844.622478591533,109,QUESTION_5,Navi,3,2 +RESPONSE_466,5.949639080864673,-0.08639413276208396,-0.2593863891078606,-0.3089133981953573,-0.10496519908766946,4.482364487914066,1,-8870.916761628036,365,QUESTION_5,Esperanto,4,4 +RESPONSE_467,5.273541313796364,-0.09985992238489158,-0.10676693157636329,0.00025940499159534584,-0.07578691961579155,4.628509527885896,18,-9404.04781775593,352,QUESTION_5,Vulcan,4,4 +RESPONSE_468,5.843567230724241,-0.07141853181521936,-0.07841061132171213,-0.0018982041234114494,-0.08643447775477223,4.342580307912778,3,-9370.059787081394,660,QUESTION_5,Esperanto,3,4 +RESPONSE_469,5.831888921305181,-0.07256683135724132,-0.13008170585037432,-0.19647790950515956,-0.10347599396382601,4.4440555177701935,9,-10723.969071135321,661,QUESTION_5,Esperanto,4,4 +RESPONSE_470,5.023069096934704,-0.1038153221361329,-0.28824421267438427,0.000410756695428805,-0.09501186624556657,4.347185039469054,1,-12643.55646335169,437,QUESTION_5,Vulcan,4,3 +RESPONSE_471,4.250469764641417,-0.09646122634585795,-0.20628464198497015,-0.4246640190460642,-0.18998151752459994,4.541462757714935,7,-6519.886194118011,142,QUESTION_5,Vulcan,2,2 +RESPONSE_472,5.108774646992911,-0.14282816060982004,-0.18136699898055422,-0.30759502892858204,-0.10830287004261964,4.646768026115124,12,-9451.622958310027,331,QUESTION_5,Klingon,4,4 +RESPONSE_473,5.050161393671974,-0.13212309387680185,-0.2331840971489592,-0.37694199868210193,-0.07791346922428549,4.257055577042079,20,-8031.332837859822,330,QUESTION_5,Vulcan,3,4 +RESPONSE_474,4.831601001554706,-0.14338718176779564,-0.29696375992035173,-0.5081635094609152,-0.18192726448940297,4.845534309855244,16,-7011.423249629852,124,QUESTION_5,Esperanto,2,2 +RESPONSE_475,5.0429520481398304,-0.09360748281808923,-0.19895708879013577,-0.3290591261984409,-0.12924454421034912,4.056647774198175,10,-9301.763056214637,312,QUESTION_5,Klingon,3,3 +RESPONSE_476,5.2715596599939465,-0.06613837467976436,-0.2552207472714188,0.0008190412764187295,-0.08804503590914176,4.572303516853885,10,-11598.740216903649,393,QUESTION_5,Klingon,4,4 +RESPONSE_477,5.860276327145631,-0.07530354761505685,-0.0639261760779183,-0.05204031763045425,-0.09074551174573467,4.696254678313837,2,-13872.854698992322,736,QUESTION_5,Navi,4,5 +RESPONSE_478,5.330620030078883,-0.12855941299208126,-0.18884216718120875,-0.2232823090331404,-0.1606110977053612,4.629621142418311,17,-11329.351544588686,398,QUESTION_5,Klingon,3,4 +RESPONSE_479,4.9076086025516945,-0.08839706400517537,-0.17706546499199521,-0.26246834108345385,-0.1066539770219095,4.407804397114447,8,-4276.129496907152,266,QUESTION_5,Vulcan,4,4 +RESPONSE_480,4.6199927541043895,-0.12387272914801199,-0.05585430033547443,-0.31725435621692605,-0.08089802900339915,4.542608309415916,2,-10147.36444392594,314,QUESTION_5,Esperanto,4,4 +RESPONSE_481,3.986511077324802,-0.2454564848559047,-0.23572105209871722,-0.5234249702662788,-0.19160679977223313,3.2598318292530575,8,-6234.0563764965045,54,QUESTION_5,Esperanto,1,1 +RESPONSE_482,5.284042681230532,-0.07348133593985726,-0.09124806717684289,-0.16287870356714143,-0.05197223627195463,4.708276234253139,2,-8662.077906030654,370,QUESTION_5,Esperanto,4,4 +RESPONSE_483,5.146233560450788,-0.08460518744571921,-0.18802877389569536,-0.17622311722355002,-0.12799638930408402,4.275243299726654,6,-10353.945897343676,361,QUESTION_5,Vulcan,4,4 +RESPONSE_484,5.379216819837475,-0.10350958413792348,-0.20048419367279677,0.0002727442779263435,-0.09411357220564587,4.240498673549288,17,-4910.705923244557,449,QUESTION_5,Klingon,4,4 +RESPONSE_485,5.190243891527393,-0.09570524832015151,-0.17261752256642535,-0.05219567277880104,-0.07387102117360105,4.5914951593995506,10,-8337.360001021207,369,QUESTION_5,Esperanto,4,4 +RESPONSE_486,6.427077665097452,-0.09001367992519567,-0.15319505010130333,-0.2731098412944186,-0.043334295256882996,4.331180014984067,16,-12317.253859215325,526,QUESTION_5,Klingon,4,4 +RESPONSE_487,5.2591222514487805,-0.07074623846201353,-0.21149296992766017,-0.20403270900215398,-0.08891841308135569,4.288141821382447,9,-6354.928618177304,378,QUESTION_5,Navi,3,3 +RESPONSE_488,6.107875974483694,-0.11359722039876503,-0.14694000877680083,-0.094186541087446,-0.09510436032275982,4.774527118719775,14,-5645.9718128995,877,QUESTION_5,Klingon,5,6 +RESPONSE_489,5.106667516585181,-0.07192438699415174,-0.2416051998210694,-0.1740152496631291,-0.05587160095092645,4.537819606656575,13,-4782.691053364309,321,QUESTION_5,Navi,3,3 +RESPONSE_490,5.242280432192564,-0.1239777193240486,-0.2257335381740454,-0.3738743690524632,-0.1479229750992305,4.376333381840351,13,-9350.357788853828,181,QUESTION_5,Navi,3,3 +RESPONSE_491,5.623862697896435,-0.10756477494937544,-0.05982781402145923,0.001455457201183773,-0.060822467419626165,4.632005902566701,8,-5402.0993561329005,540,QUESTION_5,Klingon,4,4 +RESPONSE_492,5.380563583047489,-0.10263322054904142,-0.12136285530957602,-0.27507935740542194,-0.07092134058325696,4.44791475031162,7,-12090.60083908746,401,QUESTION_5,Navi,4,4 +RESPONSE_493,5.396085248069736,-0.12224581283750444,-0.19206860391512987,-0.2883885377586882,-0.1158110011438396,4.1731893411113825,6,-6573.583650215445,220,QUESTION_5,Vulcan,2,3 +RESPONSE_494,5.731750482886787,-0.06782305886377084,-0.09789396068412513,-0.2507608348493316,-0.08975583331487352,4.320687554008289,7,-6926.272332578563,605,QUESTION_5,Vulcan,4,4 +RESPONSE_495,4.473467759779446,-0.1367911447815029,-0.22564624990973406,-0.36251649380052414,-0.15066923749106795,4.011620702925992,19,-5802.444481018318,178,QUESTION_5,Vulcan,1,2 +RESPONSE_496,5.186379408536163,-0.11907677936159303,-0.3231453680535416,-0.31210463561102314,-0.15602945119281883,3.942257068725908,10,-9368.348020413534,371,QUESTION_5,Navi,3,3 +RESPONSE_497,5.161995799446461,-0.09204146767122466,-0.1827088058264155,-0.2789209859115068,-0.12449855462349889,4.631220623877976,11,-10581.165259523721,320,QUESTION_5,Navi,3,3 +RESPONSE_498,5.216499152855678,-0.10497461900724904,-0.26497582647054213,-0.3604178541667857,-0.07435669113614467,3.829803663535034,1,-11558.060892196958,355,QUESTION_5,Klingon,4,4 +RESPONSE_499,5.450414618326595,-0.092950292244984,-0.0744692867658657,-0.22552058466419356,-0.09769852929840302,4.060003147481292,3,-7537.519938749163,530,QUESTION_5,Vulcan,3,3 +RESPONSE_500,4.501384469264368,-0.11396355781592712,-0.20214456496665706,-0.20033349972539008,-0.10260133054397946,4.853280883040842,11,-13106.598399362358,198,QUESTION_5,Klingon,4,3 diff --git a/paper.bib b/paper.bib new file mode 100644 index 000000000..9f64ff281 --- /dev/null +++ b/paper.bib @@ -0,0 +1,42 @@ +@article{Williamson2012, +author = {Williamson, David M. and Xi, Xiaoming and Breyer, F. Jay}, +journal = {Educational measurement: issues and practice}, +number = {1}, +pages = {2--13}, +title = {{A framework for evaluation and use of automated scoring}}, +volume = {31}, +year = {2012} +} + +@online{github_rsmtool, + author = {Madnani, Nitin and Loukina, Anastassia}, + title = {RSMTool}, + year = 2016, + url = {https://github.com/EducationalTestingService/rsmtool}, + urldate = {2016-06-23} +} + +@misc{dan_blanchard_2014_12825, + author = {Dan Blanchard and + Nitin Madnani and + Michael Heilman and + Nils Murrugarra Llerena and + Diane M. Napolitano and + Aoife Cahill and + Keelan Evanini and + Chee Wee Leong}, + title = {SciKit-Learn Laboratory (SKLL) 1.0.0}, + month = nov, + year = 2014, + doi = {10.5281/zenodo.12825}, + url = {http://dx.doi.org/10.5281/zenodo.12825} +} + +@inproceedings{Loukina2015, +address = {Denver, Colorado}, +author = {Loukina, Anastassia and Zechner, Klaus and Chen, Lei and Heilman, Michael}, +booktitle = {Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications}, +pages = {12--19}, +title = {{Feature selection for automated speech scoring}}, +year = {2015} +} diff --git a/paper.json b/paper.json new file mode 100644 index 000000000..09bae4fa7 --- /dev/null +++ b/paper.json @@ -0,0 +1,30 @@ +{ + "@context": "https://raw.githubusercontent.com/mbjones/codemeta/master/codemeta.jsonld", + "@type": "Code", + "author": [ + { + "@id": "0000-0001-9354-6851", + "@type": "Person", + "email": "nmadnani@ets.org", + "name": "Nitin Madnani", + "affiliation": "Educational Testing Service" + }, + { + "@id": "0000-0002-8213-3362", + "@type": "Person", + "email": "aloukina@ets.org", + "name": "Anastassia Loukina", + "affiliation": "Educational Testing Service" + } + ], + "identifier": "", + "codeRepository": "https://github.com/EducationalTestingService/rsmtool", + "datePublished": "2016-06-27", + "dateModified": "2016-06-27", + "dateCreated": "2016-06-27", + "description": "RSMTool is a collection of tools for researchers working on statistical models for automated scoring of written and spoken responses in an educational context.", + "keywords": "educational applications, scoring models, statistics, visualization", + "license": "Apache 2.0", + "title": "RSMTool", + "version": "5.0.2" +} diff --git a/paper.md b/paper.md new file mode 100644 index 000000000..638e928bd --- /dev/null +++ b/paper.md @@ -0,0 +1,27 @@ +--- +title: 'RSMTool: collection of tools building and evaluating automated scoring models' +tags: + - educational applications + - scoring models + - statistics + - visualization +authors: + - name: Nitin Madnani + orcid: 0000-0001-9354-6851 + affiliation: Educational Testing Service + - name: Anastassia Loukin + orcid: 0000-0002-8213-3362 + affiliation: Educational Testing Service +date: 27 June 2016 +bibliography: paper.bib +--- + +# Summary + +RSMTool is a collection of tools for researchers working on the development of automated scoring engines for written and spoken responses in an educational context. The main purpose of the tool is to simplify the integration of educational measurement recommendations into the model building process and to allow the researchers to rapidly and automatically generate comprehensive evaluations of the scoring model. + +RSMTool takes as input a feature file with numeric, non-sparse features extracted from the responses and a human score and lets you try several different machine learning algorithms to try and predict the human score from the features. RSMTool allows the use of simple OLS regression as well as several more sophisticated regressors including Ridge, SVR, AdaBoost, and Random Forests, available through integration with the SKLL toolkit [@skll]. The tool also includes several regressors which ensure that all coefficients in the final model are positive to meet the requirement that all feature contributions are additive [@loukina2015]. The primary novel contribution of RSMTool is a comprehensive, customizable HTML statistical report that contains feature descriptives, subgroup analyses, model statistics, as well as several different evaluation measures illustrating model efficacy. The various numbers and figures in the report are highlighted based on whether they exceed or fall short of the recommendations laid out by [@Williamson2012]. The structure of RSMTool makes it easy for researchers to add new analyses without making any changes to the core code structure thus allowing for a wide range of psychometric evaluations. + +The tool is written in Python and works on all platforms (Windows/Linux/Mac OS X). The source code is available from Github [@github_rsmtool] + +# References diff --git a/rsmtool/notebooks/builtin_model.ipynb b/rsmtool/notebooks/builtin_model.ipynb index 6b527b52c..d3b4cd044 100644 --- a/rsmtool/notebooks/builtin_model.ipynb +++ b/rsmtool/notebooks/builtin_model.ipynb @@ -82,7 +82,7 @@ "\n", "The relative coefficients are intended to show relative contribution of different feature and their primary purpose is to indentify whether one of the features has an unproportionate effect over the final score. They are computed as standardized/(sum of absolute values of standardized coefficients). \n", "\n", - "Negative standardized coefficients are highlighted in red.\n", + "Negative standardized coefficients are highlighted.\n", "\n", "**Note**: if the model contains negative coefficients, relative values will not sum up to one and their interpretation is generally questionable. " ] diff --git a/rsmtool/notebooks/comparison/header.ipynb b/rsmtool/notebooks/comparison/header.ipynb index 81bab5d50..5bcb7b140 100644 --- a/rsmtool/notebooks/comparison/header.ipynb +++ b/rsmtool/notebooks/comparison/header.ipynb @@ -36,6 +36,7 @@ "import logging\n", "import numpy as np\n", "import os\n", + "import platform\n", "import pandas as pd\n", "import re\n", "import sys\n", @@ -44,49 +45,56 @@ "from os.path import exists, join\n", "\n", "from IPython import sys_info\n", - "from IPython.display import display, HTML, Image, Markdown, SVG" + "from IPython.display import display, HTML, Image, Markdown, SVG\n", + "from rsmtool.version import VERSION as rsmtool_version" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "" ] }, diff --git a/rsmtool/notebooks/comparison/sysinfo.ipynb b/rsmtool/notebooks/comparison/sysinfo.ipynb index e52073210..d75d0c13d 100644 --- a/rsmtool/notebooks/comparison/sysinfo.ipynb +++ b/rsmtool/notebooks/comparison/sysinfo.ipynb @@ -7,6 +7,32 @@ "## System information" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "system_name = platform.system()\n", + "\n", + "# People might not know what 'Darwin' is, so we should replace that with 'Mac OS X'\n", + "if system_name == 'Darwin':\n", + " system_name = 'Mac OS X'\n", + " \n", + "# get the architecture\n", + "architecture = platform.architecture()[0]\n", + "\n", + "# get the rsmtool version\n", + "rsmtool_version_str = '.'.join(map(str, rsmtool_version))\n", + "\n", + "display(Markdown('This report was generated using rsmcompare v{} on a '\n", + " '{} computer running {}.'.format(rsmtool_version_str, \n", + " architecture, \n", + " system_name)))" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -23,7 +49,8 @@ "outputs": [], "source": [ "import pip\n", - "sorted([\"%s==%s\" % (i.key, i.version) for i in pip.get_installed_distributions()])" + "package_names = '\\n'.join(sorted([\"%s==%s\" % (i.key, i.version) for i in pip.get_installed_distributions()]))\n", + "display(HTML('
{}
{}