Creating an Online Judge: Preparation

December 30, 2019

Motivated by the CafeCoder project, I want to try to build a prototype of a minimal online judge using NodeJS. I have always wanted to build an online judge myself, and I hope that the project would be robust enough to support contests and regular problem testing.


I hope that the system would be scalable to support 200 to 300 concurrent users in competition (of course, not submitting at the same time). This is not a size that is too difficult to deal with, which simplifies some of the details we will talk about later.


Before starting, we should decide how the data required for the online judge to operate will be stored.

  • User information: the email, username and password (hashed with salt) will be stored in the database. We will not (yet?) implement OpenID login to simplify the development.
  • Problem information (like problem ID, title, time limit, memory limit, description, etc, excluding test data): these information will also be stored in database so they can be retrieved easily to render the website.
  • Test data: these will be stored in hard disk and we will allow the judgers to retrieve the files when needed.


Sandboxing is needed for both the compilation process and the execution process. We will use ioi/isolate for this purpose.


The system will consist one main server and at least one (possibly multiple) judgers.

The main server will have two HTTP APIs, one exposed to the public and one exposed to judgers only.

To connect the judgers to the main server, we will first generate a pair of judge name and password on the main server, most preferably using an admin user interface. After that, we will set the judge name and password to the judger, which will then connect to the main server. The judger would then use this judge name and password every time it has to retrieve information from or put information on the main server.

At the end, we want to simplify the scaling process, for example, when we use Amazon Lightsail. We would package both the main server (optional) and the judger into Docker, then possibly create a launch script that we can use to spawn multiple judgers before contests.

The website that is rendered will be written in EJS, so it can be easily edited to adapt to the need of who is hosting the judge.

We may need a queue for pending submissions that are not judged yet.