Getting started

What is Baygon?

Baygon is a minimalistic test framework for any types of executables. It provides a simple way of testing code with a JSON or a YAML description of tests.

It is mainly designed for simple tests in academic environments, but it can be used for any kind of tests.

Points can be assigned to tests, group of tests or automatically distributed based on the number of tests. The total earned points can be used to calculate the final assignment grade.

Warning

Baygon is currently in beta stage. It's ready to be used for building functional tests, but the config and API are not stable enough, which is likely to have breaking changes between minor releases.

Have fun squashing bugs!

How it works

Baygon is a CLI tool that runs tests described in a JSON or YAML file. It can be used to test any kind of executable, including binaries, scripts, and even web applications. It's designed to be used for student assignments.

Baygon first validates the configuration file and converts it into an immutable SuiteModel. The runtime then walks that model, inherits shared settings such as filters and executables, and executes each case sequentially.

By default Baygon will run all the tests in the description file.

Get started

Let's say you have a C program you want to test:

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    if (argc > 1 && strcmp(argv[1], "--version") == 0) {
        fprintf(stderr, "Version 0.1.1\n");
        return 0;
    }
    if (argc != 2 + 1) return 1;
    printf("%d", atoi(argv[1]) + atoi(argv[2]));
}

You then need to write a test file that can be named t.yaml, t.json, test.yaml, test.json, tests.yml or test.json. Baygon is gonna find it. Inside this file you describe each individual functional test:

version: 1
tests:
  - name: Arguments check
    tests:
      - name: No errors if two arguments
        args: [1, 2]
        exit: 0
      - name: Error if less than two arguments
        args: [1]
        exit: 1
  - name: Stdout is the sum of arguments
    args: [1, 2]
    stdout:
      - equals: "3"
  - name: Version on stderr
    args: ['--version']
    stderr:
      - match: 'm/\b\d\.\d\.\d\b/'
      - contains: 'Version'

To be able to run the tests, simply install Baygon:

pip install baygonDone!

Then build and test you application:

# Compile the C programcc app.c -o a.out# Run baygon testsbaygon -v ./a.outTest 1: Arguments check
Test 1.1: No errors if two arguments.......... PASSED
Test 1.2: Error if less than two arguments.... PASSED
Test 2: Stdout is the sum of arguments.......... PASSED
Test 3: Version on stderr....................... PASSED

Ran 4 tests in 0.01s.

ok.

Explore the CLI output

Baygon offers a few handy flags to tailor the feedback you receive during a run:

• --pretty formats failing tests as rich panels, highlighting issues and the commands that were executed.
• -T/--table prints a summary table once the run finishes, so you can quickly scan the status and points of each test.

Tip

You may need to use pip3 instead of pip depending on your system.