v1.2.0 bump + release notes + docs (#109)

README.md

@@ -1,4 +1,4 @@
-# tcframe 1.1.0
+# tcframe 1.2.0
 
 [![GitHub Release](https://img.shields.io/github/release/tcframe/tcframe.svg)](https://github.com/tcframe/tcframe)
 [![Build Status](https://img.shields.io/travis/tcframe/tcframe/master.svg)](https://travis-ci.org/tcframe/tcframe)

docs/api-ref/api-ref.rst

@@ -225,6 +225,38 @@ The following macros are exposed to define input/output formats:
 
 ----
 
+.. _api-ref_styles:
+
+Problem styles
+**************
+
+.. sourcecode:: cpp
+
+    virtual void StyleConfig() {}
+
+Defines the options to enable for problem styles. The following methods are exposed:
+
+.. cpp:function:: CustomScorer()
+
+    Declares that the problem needs a custom scorer.
+
+.. cpp:function:: NoOutput()
+
+    Declares that the problem does not need test case output files.
+
+See :ref:`styles` for more details.
+
+Example:
+
+.. sourcecode:: cpp
+
+    void StyleConfig() {
+        CustomScorer();
+        NoOutput();
+    }
+
+----
+
 .. _api-ref_constraints:
 
 Constraints and subtasks
@@ -524,6 +556,10 @@ Test cases generation
 
     The solution command to use for generating output files. Default: ``./solution``.
 
+.. py:function:: --scorer=<command>
+
+    The custom scorer command to use for checking sample output strings in problem spec class. Default: ``./scorer``.
+
 .. py:function:: --seed=<seed>
 
     The seed for random number generator ``rnd`` in the test spec. Default: ``0``.
@@ -543,6 +579,9 @@ Local grading
 
     The solution command to grade. Default: ``./solution``.
 
+.. py:function:: --scorer=<command>
+
+    The custom scorer command to use. Default: ``./scorer``.
 
 .. py:function:: --time-limit=<time-limit-in-seconds>
 

docs/conf.py

@@ -53,9 +53,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.1'
+version = '1.2'
 # The full version, including alpha/beta/rc tags.
-release = '1.1.0'
+release = '1.2.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/getting-started/getting-started.rst

@@ -44,7 +44,7 @@ Installation
 
 Firstly, we must get **tcframe** on our system. It consists of C++ header files and a few helper scripts.
 
-Download the latest **tcframe** `here <https://github.com/tcframe/tcframe/releases/download/v1.1.0/tcframe_1.1.0.zip>`_. Extract the zip file somewhere on your system; for example, ``~/tcframe``. We will call this directory "**tcframe** home". Confirm that you extracted it correctly by verifying that the directory ``include`` exists directly inside **tcframe** home.
+Download the latest **tcframe** `here <https://github.com/tcframe/tcframe/releases/download/v1.2.0/tcframe_1.2.0.zip>`_. Extract the zip file somewhere on your system; for example, ``~/tcframe``. We will call this directory "**tcframe** home". Confirm that you extracted it correctly by verifying that the directory ``include`` exists directly inside **tcframe** home.
 
 Then, add the following lines to your ``~/.bashrc``. This will set the environment variable ``TCFRAME_HOME`` to our **tcframe** home directory, and make ``tcframe`` command available everywhere.
 

docs/release-notes/1_2_0.rst

@@ -0,0 +1,19 @@
+.. _v1_2_0:
+
+1.2.0
+=====
+
+December 16, 2016
+
+Bugfixes
+--------
+
+- Fixed a regression when the final verdict of local grading of problems without subtasks is always AC.
+
+New features
+------------
+
+- It is now possible to write a custom program that checks whether the contestant's output is correct (instead of just comparing it with the official output). This is called custom scorer program.
+- It is now possible not to generate test case output (``.out``) files.
+
+See :ref:`styles` for details on how to enable the above options.

docs/release-notes/release-notes.rst

@@ -6,6 +6,7 @@ Release Notes
 .. toctree::
    :maxdepth: 1
 
+   1_2_0
    1_1_0
    1_0_1
    1_0_0

docs/topic-guides/grading.rst

@@ -39,6 +39,9 @@ Runtime Error (RTE)
 Time Limit Exceeded (TLE)
     The solution did not stop within the time limit, if specified.
 
+Internal Error (ERR)
+    Custom scorer (if any) crashed or did not give valid verdict.
+
 The verdict of each subtask will be also shown. The verdict of a subtask is the worst verdict of all verdicts of test cases that are assigned to it. Here, RTE is worse than WA, and WA is worse than AC.
 
 Here is a sample output of a local grading for problems with subtasks.

docs/topic-guides/styles.rst

@@ -0,0 +1,84 @@
+.. _styles:
+
+Problem Styles
+==============
+
+Currently, **tcframe** supports only **batch**-style problems, where the solution is expected to read the test cases from the standard input and write the answers to the standard output. There are some configurable options to this behavior, which can be specified in the ``StyleConfig()`` method of the problem spec class.
+
+.. sourcecode:: cpp
+
+    void StyleConfig() {
+        // option specifications
+    }
+
+The available options are as follows.
+
+Custom scorer
+-------------
+
+Enabled by calling ``CustomScorer()`` inside ``StyleConfig()``.
+
+A scorer is a program which decides the verdict of a test case. By default, the scorer is the simple ``diff`` program. If custom scorer is enabled, then you must provide the custom scorer program.
+
+The custom scorer will receive the following arguments:
+
+- argv[1]: test case input filename
+- argv[2]: test case output filename
+- argv[3]: contestant's produced output filename
+
+The custom scorer must print the test case verdict to the standard output, which is a line consisting of either:
+
+- ``AC``: indicates that the contestant's output is correct
+- ``WA``: indicates that the contestant's output is incorrect
+
+The custom scorer must be compiled prior test cases generation/local grading, and the execution command should be passed to the runner program as the ``--scorer`` option. For example:
+
+::
+
+    ./runner grade --solution=./solution_alt --scorer=./my_custom_scorer
+
+The default scorer command is ``./scorer`` if not specified.
+
+Here is an example custom scorer which gives AC if the contestant's output differs not more than 1e-9 with the official output.
+
+.. sourcecode:: cpp
+
+    #include <bits/stdc++.h>
+    using namespace std;
+
+    int wa() {
+        cout << "WA" << endl;
+        return 0;
+    }
+
+    int ac() {
+        cout << "AC" << endl;
+        return 0;
+    }
+
+    int main(int argc, char* argv[]) {
+        ifstream tc_in(argv[1]);
+        ifstream tc_out(argv[2]);
+        ifstream con_out(argv[3]);
+
+        double tc_ans;
+        tc_out >> tc_ans;
+
+        double con_ans;
+        if (!(con_out >> con_ans)) {
+            return wa();
+        }
+
+        if (abs(tc_ans - con_ans) < 1e-9) {
+            return ac();
+        } else {
+            return wa();
+        }
+    }
+
+No output
+---------
+
+Enabled by calling ``NoOutput()`` inside ``StyleConfig()``.
+
+Sometimes, a problem does not need test case output files (``.out``) because the scoring is done by a custom score alone. If this option is enabled, then ``.out`` files will not be generated, and it is not allowed to specify ``Output()`` in sample test cases.

docs/topic-guides/test-cases.rst

@@ -134,6 +134,8 @@ can be translated to:
 
     The ``Output()`` part of a sample test case definition is optional, and if not present, the solution will be run to produce the output. However, it is only for easier migration from **tcframe** 0.x. You should always specify both input and output, so that you are sure you are typing the output correctly in the problem statement by only looking at the spec file (no need to check with the actual produced output file).
 
+    Of course, if ``NoOutput()`` is enabled (see :ref:`styles`), then ``Output()`` is not allowed to be specified.
+
 :ref:`If your problem has subtasks <subtasks>`, you also need to assign each sample test case to a set of subtasks, by calling ``Subtasks()`` at the beginning of each ``SampleTestCaseX()`` with the set of subtask numbers, as follows.
 
 .. sourcecode:: cpp

docs/topic-guides/topic-guides.rst

@@ -12,6 +12,7 @@ In this guide, we will learn each aspect of **tcframe** more thoroughly. Each se
    Spec and runner <spec>
    I/O variables <io-variables>
    I/O formats <io-formats>
+   Problem styles <styles>
    Constraints <constraints>
    Subtasks <subtasks>
    Test cases <test-cases>

scripts/tcframe

@@ -21,7 +21,7 @@ build() {
 }
 
 version() {
-    echo "tcframe 1.1.0"
+    echo "tcframe 1.2.0"
 }
 
 if [ -z "$TCFRAME_HOME" ]; then