Aaryak

DOCS.md

@@ -0,0 +1,98 @@
+
+# RCE Documentation
+
+## Setting up the application
+
+### Installation:
+
+The application can be installed by cloning this repository.
+The project requires the host machine to have docker and docker-compose installed. Follow the official installation guide available at [Install Docker Engine](https://docs.docker.com/engine/install/ubuntu/) and [Install Docker Compose](https://docs.docker.com/compose/install/).
+Note that docker-compose depends on docker engine to function.
+
+### Permissions:
+
+Docker may return a Permission Denied error without being run as `sudo`. To fix this, we do the following:
+```
+$ sudo groupadd docker
+$ sudo usermod -aG docker $USER
+$ newgrp docker
+$ docker run hello-world
+```
+Reboot the host machine.
+
+## Starting the application
+Start a terminal and redirect into the project repository:
+```
+$ cd /path/to/repo/remote-code-executor
+```
+
+### Using the Bash Script [Linux/Unix only]:
+
+Run the provided `start.sh` bash script:
+```
+$ ./start.sh
+Select mode [P]roduction / [d]evelopment: d
+```
+
+Providing input as `d` starts the application in development mode, and reflects all the changes made before starting. Providing `p` as the input starts the application without rebuilding, any changes made will not be reflected.
+
+The mode can directly be provided to the command as an argument:
+```
+$ ./start.sh p
+```
+
+### Using Docker Commands:
+
+The project uses a docker volume called `userdata` to store any user generated files [code, input, a.out files]. We can create this volume as follows:
+```
+$ docker volume create --name=userdata
+```
+The rest of the setup is handled by docker-compose:
+```
+$ docker-compose up
+```
+The containers are built from scratch only for the first time we run this command. Any subsequent attempts will simply start the existing container. To rebuild the containers (in case of any changes to the source), we can run:
+```
+$ docker-compose up --build
+```
+
+## How it works
+
+> NOTE: This project currently only has a server API. The front-end for the same is not yet available. Postman or cURL can be used to run the application meanwhile.
+
+The back-end functionality is provided by means of two docker services that run node applications -- `server` and `executor`, each in its own container. The containers are linked by the `userdata:/storage` mounted volume, as well as over a network, thanks to docker-compose.
+
+Starting the application in any of the above listed methods starts both of these services simultaneously. The `server` listens on port 9000, while the `executor` listens on port 8080.
+
+### The Executor:
+
+`executor` is an express server API that accepts a POST request at `/code/<lang>`, where the language is simply the file extension used by that programming language (py, cpp, js).
+
+The request body contains a single key `filePath` which is the path to the user's files in the mounted volume `/storage/`. These files are generated by the `server` container and will be explained later.
+
+This service simply executes the code file at the given location and returns the result as a JSON response.
+All user files generated for this execution are deleted by this service.
+
+### The Server:
+
+`server` is also an express API. The client can send a POST request to the `/code` endpoint.
+The request body here contains 4 keys:
+-  `key <String>` - A random string to uniquely identify each request
+
+-  `language <String>` - Represents the programming language to be used. Value is identical to the file extension used by the language
+
+-  `code <String>` - User provided code
+
+-  `input <String>` - User provided input for their code
+
+`key` can be entered manually as any random string using Postman or cURL. It can also be obtained by sending a GET request to `/code`. This system is in place for the front-end to utilise in the future.
+
+**Significance of the key:**
+The `key` is the string that acts as the name of all the files a user's request generates. For example, if a user sends a C++ program along with a key of `abcde` then the following files are generated:
+```
+storage
+├── abcde.cpp		(source code file)
+├── abcde		(input text file)
+└── abcde.out		(output file, generated on compilation)
+```
+The `key` is thus also used to generate the `filePath` variable that we saw in the executor service. The `filePath` is simply given (from root) as -- `/storage/${key}` without any file extension.

README.md

@@ -4,19 +4,24 @@ This RCE project currently provides a basic API to execute programs on a remote
 
 ## Functioning:
 
-The API accepts POST requests to the `/code` endpoint.
+The API accepts POST requests to the `localhost:9000/code` endpoint. Check out DOCS.md for a detailed reference
 
 ### Request Body Keys:
 
-- Key- A random string to uniquely identify each request
-- Languge
-- Code String
-- Input String
+- `key <String>` - A random string to uniquely identify each request
+
+- `language <String>` - Represents the programming language to be used. Value is identical to the file extension used by the language
+
+- `code <String>` - User provided code
+
+- `input <String>` - User provided input for their code
 
 ### Supported Languages:
 
 - Python3 (use: _py_)
+
 - C++ (use: _cpp_)
+
 - Node/Javascript (use: _js_)
 
 ## Code Validation:
@@ -26,37 +31,63 @@ Currently, code validation takes place by selectively rejecting or accepting lib
 #### Python (Rejected Libraries):
 
 - os
+
 - subprocess
+
 - shlex
+
 - xml
+
 - pickle
 
 #### C++ (Accepted Libraries):
 
 - iostream
+
 - algorithm
+
 - stdio
+
 - cstdio
+
 - vector
+
 - math
+
 - cmath
+
 - cstring
+
 - string
+
 - deque
+
 - iomanip
+
 - iterator
+
 - map
+
 - queue
+
 - set
+
 - stack
+
 - conio
+
 - ctype
 
 #### Node/Javascript (Accepted Libraries):
 
 - readline
+
 - buffer
+
 - string_decoder
+
 - timers
+
 - stream
-- util
+
+- util

docker-compose.yml

@@ -0,0 +1,24 @@
+version: "3.8"
+services:
+  # server container manages client requests
+  server:
+    build: ./server
+    ports:
+      - "9000:9000"
+      # localhost:9000/ for host machine
+      # server:9000/ for containers
+    volumes: 
+      - userdata:/storage # mounted volume, contains user generated data
+  # executor container responds to requests from the server container
+  executor:
+    build: ./executor
+    ports:
+      - "8080:8080"
+      # localhost:8080/ for host machine
+      # executor:8080/ for containers
+    volumes: 
+      - userdata:/storage # mounted volume, contains user generated data
+volumes:
+  # userdata volume common to all composed containers
+  userdata:
+    external: true

executor/.dockerignore

@@ -0,0 +1 @@
+node_modules/

executor/Dockerfile

@@ -0,0 +1,17 @@
+FROM node:14-alpine
+
+WORKDIR /executor
+
+RUN apk add python3
+
+RUN apk add g++
+
+ENV NODE_ENV=production
+
+COPY ["package.json", "package-lock.json*", "./"]
+
+RUN npm install --production
+
+COPY . .
+
+CMD ["node", "app.js"]

executor/app.js

@@ -0,0 +1,16 @@
+const express = require('express')
+const runCode = require('./lib/runCode')
+const bodyParser = require('body-parser')
+
+const PORT = 8080
+const app = express()
+
+app.get('*', (req, res) => {
+  res.status(404).send('API does not support this endpoint')
+})
+
+app.use('/code', require('./routes/code'))
+
+app.listen(PORT, () => {
+  console.log('Executor available on port 8080')
+})

executor/lib/runCode.js

@@ -0,0 +1,56 @@
+const util = require('util')
+const exec = util.promisify(require('child_process').exec)
+const fs = require('fs')
+
+const runCode = async (language, filePath) => {
+  console.log('runCode called')
+  var result = { stderr: 'no output' }
+  switch (language) {
+    case 'cpp': {
+      console.log('c++ exec')
+      try {
+        result = await exec(
+          `g++ -o ${filePath}.out ${filePath}.cpp && ${filePath}.out < ${filePath}`,
+          { timeout: 3000, maxBuffer: 1024 * 1024 * 5 }
+        )
+        fs.promises.unlink(filePath + '.out')
+      } catch (err) {
+        result = { stderr: err }
+      }
+      break
+    }
+    case 'js': {
+      console.log('js exec')
+      try {
+        result = await exec(`node ${filePath}.js < ${filePath}`, {
+          timeout: 5000,
+          maxBuffer: 1024 * 1024 * 5,
+        })
+      } catch (err) {
+        result = { stderr: err }
+      }
+      break
+    }
+    case 'py': {
+      console.log('runCode.js py switch')
+      console.log('executing')
+      try {
+        result = await exec(`cat ${filePath} | python3 ${filePath}.py`, {
+          timeout: 5000,
+          maxBuffer: 1024 * 1024 * 5,
+        })
+      } catch (err) {
+        result = { stderr: err }
+      }
+      break
+    }
+  }
+
+  fs.promises.unlink(filePath)
+  fs.promises.unlink(filePath + '.' + language)
+  // console.log('returning from runCode() with result');
+  // console.log(result);
+  return result
+}
+
+module.exports = runCode