Changelog¶
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Unreleased¶
0.9.0 (2026-11-16)¶
Added¶
numbermatcher: the output is a number equal to a value, with an optionaltolerance, or withinmin/max(#27)lengthandlinesmatchers: number of characters or lines of the output, exact or within a range (#27)stdincan be the output of a generator program:stdin: {executable, args}(#30)same-asmatcher: compare an output with the output of a reference program run with the same arguments and input (#30)filterscan be set on a test or a group, as the documentation said- A bare executable name such as
grepis looked up in thePATHwhen it is not a file next to the configuration - Documentation: matchers, expressions (
eval), and a page of recipes to check the source code (#21) - Claude Code skill to write and run Baygon tests, available as a plugin (
claude plugin marketplace add heig-tin-info/baygon) and from the package withbaygon --install-skill user|project
Changed¶
- Exit status failures read
Invalid exit status: expected 1, got 0.instead of1 != 0
Fixed¶
regexandreplacefilters given in the configuration crashed the run- Summary percentage without float noise (
14.29% ok) - Documentation examples that were invalid (YAML syntax, unknown
matchmatcher,stdinused as a file name) - Documentation:
trimremoves whitespace at both ends of the output, not on each line
0.8.0 (2026-11-16)¶
Added¶
- Support for Python 3.14 and 3.15
timeoutkey (seconds) at suite, group and test level, and--timeoutoption: a program that hangs is killed, with the processes it started, and only its test fails (#47)max-outputkey (bytes, or sizes such as64K): a program that prints without end is killed once a stream goes over the limit. Output is capped at 16 MiB by default (#51)- The report written by
--reportlists every test (cases) and a subtotal per group (groups); also available asRunReport.as_dict()(#51) --only IDand-k/--keyword TEXTrun a subset of the tests;SuiteContext.select()andbaygon.core.select_tests()do the same from Python (#51)--strictexits with status 1 when a test fails (#51)
Changed¶
- Upgrade dependencies (typer up to 0.29, rich 15);
termynalis now a docs-only dependency and the unusedeval-type-backportdependency is dropped - CI: up-to-date actions, Python 3.10 to 3.15 and macOS; a release now runs the whole CI before publishing to PyPI
- A group or a test can override an inherited
executable; the nearest one wins (#51) - Invalid configurations are reported as one line per error, without a traceback (#26)
Fixed¶
- A failed negated matcher (
not: contains) now says the output should not contain the value, instead of the opposite (#51) - A boolean filter set to
false(e.g.uppercase: false) is no longer applied - Output that is not valid UTF-8 no longer stops the run; invalid bytes are replaced
- A program that cannot be started (e.g. a script without shebang) fails its test instead of stopping the run
Executable(path, encoding=...)no longer raises aTypeErrorevalgiven as a mapping (start,end,init) no longer crashes on load (regression of 0.7.0), andeval: falseno longer enables evalevaldelimiters are literal text (start: "[["works) anditer()works inside nested calls such asiter(int(5), 1)exit: true/exit: falseare read as 1 and 0 instead of crashing the run- An empty expected output (
stdout: "",equals: "") is accepted min-pointsmust be greater than 0 (0 crashed the score computation)- An invalid regex, a non-numeric
exitor an eval error fails its test instead of stopping the run --report report.ymlwrites YAML (only.yamlwas recognised)- Point totals no longer show float noise (
0.30000000000000004) - Configuration errors point to the test case or group that is wrong, instead of both
- Documentation: the filters are applied in a fixed order; the program runs in the current directory
- Documentation: the filter is
ignorespaces(ignore-spacesis an accepted alias) (#32)
0.7.0 (2026-03-02)¶
Added¶
- Support for Python 3.13
- Adds
--report=filenameto generate a test report - Adds
--format=[json|yaml]to specify the report format - Points are reported in the test report and on the output
- CLI arguments can now be passed in the configuration file
- Adds pretty table summary using rich
- Points distribution mechanism
- Adds
baygon.ymlself-test so Baygon can validate itself with Baygon Schema()accepts YAML strings and file-like objects in addition to mappings- Changelog is now published inside the MkDocs documentation
- Adds API documentation
- Expose a programmatic runner via
baygon.runtime.BaygonRunnerand configuration helpers inbaygon.config - Provide public dataclasses in
baygon.core.modelsto represent suites, groups and cases
Changed¶
- Migration to Poetry, Ruff and Black
- Development tooling now relies on Nox sessions backed by uv groups instead of tox
- Contributor documentation now explains the uv-powered workflow for development and docs builds
- Adopted keepachangelog format
- Configuration validation now relies on Pydantic instead of Voluptuous for clearer errors
- YAML parsing errors raise
ConfigSyntaxErrorwith line and column details - Documentation migrated from VuePress to MkDocs Material with refreshed navigation
- Migrate documentation from VuePress to MkDocs Material
--configaccepts both-cand legacy-tshort flags; the summary table flag now maps to-T.- CLI rendering now delegates to dedicated presentation modules and the core runtime service
- Filters and matchers rely on explicit registries instead of module introspection
Fixed¶
- Summary table now shows failed and skipped tests with the correct status labels.
- '4 failed, 0 passed (0.0%% ok).' remove the duplicated pecentage sign
- Fix output by adding quotes and
(empty)for empty strings - Self-test now targets the repository binary and reports the resolved configuration path, so
uv run baygon $(which baygon)succeeds again - More tests (89% coverage), enabled doctests
Deprecated¶
- Drop support for Python 3.6, 3.7 and 3.8
- The legacy object wrappers (
TestSuite,TestGroup,TestCase) have been removed in favor of the service-based API (SuiteLoader,SuiteExecutor,SuiteService). - The legacy
-tshort flag for--config(still accepted in this release)
0.6.0 (2025-10-14)¶
Changed¶
- Replace the short option for
--configwith-cso-tcan be used for the summary table flag.
Fixed¶
- Restore compatibility with Click 8.3 by using boolean defaults for flags and avoiding attribute access on integers.
- Ensure report files are written when the format is omitted by selecting a format based on the filename extension.
- Reapply suite-level filters to test output so configuration filters such as
ignorespaceswork again. - Avoid passing positional arguments directly to the Click command when running
python -m baygon.
0.5.1 (2022-11-08)¶
- Add tests for filters
- Fix filters args
0.5.0 (2022-11-10)¶
- Code Refactoring
- Migrate documentation from VuePress 1.x to 2.x
- Add new features
- Add optional
pointsto config file to rank tests - Add
--jsonoption tobaygoncommand
0.2.1¶
- Use config.cfg
- Fix documentation (exit-status -> exit)
0.2.0¶
- Add negation
notkeyword - PATH is resolved for binaries