Changes in version 0.11
=======================

Released on October 23rd, 2014.

* Added support to print the details of all test cases (metadata and
  their output) to 'report'.  This is via a new '--verbose' flag which
  replaces the previous '--show-context'.

* Added support to specify the amount of physical disk space required
  by a test case.  This is in the form of a new "required_disk_space"
  metadata property, which can also be provided by ATF test cases as
  "require.diskspace".

* Assimilated the contents of all the kyua-*-tester(1) and
  kyua-*-interface(7) manual pages into more relevant places.  In
  particular, added more details on test program registration and their
  metadata to kyuafile(5), and added kyua-test-isolation(7) describing
  the isolation features of the test execution.

* Assimilated the contents of all auxiliary manual pages, including
  kyua-build-root(7), kyua-results-files(7), kyua-test-filters(7) and
  kyua-test-isolation(7), into the relevant command-specific manual
  pages.  This is for easier discoverability of relevant information
  when reading how specific Kyua commands work.

* Issue 30: Plumbed through support to query configuration variables
  from ATF's test case heads.  This resolves the confusing situation
  where test cases could only do this from their body and cleanup
  routines.

* Issue 49: Extended 'report' to support test case filters as
  command-line arguments.  Combined with '--verbose', this allows
  inspecting the details of a test case failure after execution.

* Issue 55: Deprecated support for specifying test_suite overrides on
  a test program basis.  This idiom should not be used but support for
  it remains in place.

* Issue 72: Added caching support to the getcwd(3) test in configure so
  that the result can be overriden for cross-compilation purposes.

* Issue 83: Changed manual page headings to include a "kyua" prefix in
  their name.  This prevents some possible confusion when displaying,
  for example, the "kyua-test" manual page with a plain name of "test".

* Issue 84: Started passing test-suite configuration variables to plain
  and TAP test programs via the environment.  The name of the
  environment variables set this way is prefixed by TEST_ENV_, so a
  configuration variable of the form
  'test_suites.some_name.allow_unsafe_ops=yes' in kyua.conf becomes
  'TEST_ENV_allow_unsafe_ops=YES' in the environment.

* Issue 97 and 116: Fixed the build on Illumos.

* Issue 102: Set TMPDIR to the test case's work directory when running
  the test case.  If the test case happens to use the mktemp(3) family
  of functions (due to misunderstandings on how Kyua works or due to
  the reuse of legacy test code), we don't want it to easily escape the
  automanaged work directory.

* Issue 103: Started being more liberal in the parsing of TAP test
  results by treating the number in 'ok' and 'not ok' lines as optional.

* Issue 105: Started using tmpfs instead of md as a temporary file
  system for tests in FreeBSD so that we do not leak md(4) devices.

* Issue 109: Changed the privilege dropping code to start properly
  dropping group privileges when unprivileged_user is set.  Also fixes
  testers/run_test:fork_wait__unprivileged_group.

* Issue 110: Changed 'help' to display version information and clarified
  the purpose of the 'about' command in its documentation.

* Issue 111: Fixed crash when defining a test program in a Kyuafile that
  has not yet specified the test suite name.

* Issue 114: Improved the kyuafile(5) manual page by clarifying the
  restrictions of the include() directive and by adding abundant
  examples.


Changes in version 0.10
=======================

Experimental version released on August 14th, 2014.

* Merged 'kyua-cli' and 'kyua-testers' into a single 'kyua' package.

* Dropped the 'kyua-atf-compat' package.

* Issue 100: Do not try to drop privileges to unprivileged_user when we
  are already running as an unprivileged user.  Doing so is not possible
  and thus causes spurious test failures when the current user is not
  root and the current user and unprivileged_user do not match.

* Issue 79: Mention kyua.conf(5) in the "see also" section of kyua(1).

* Issue 75: Change the rewrite__expected_signal__bad_arg test in
  testers/atf_result_test to use a different signal value.  This is to
  prevent triggering a core dump that made the test fail in some platforms.


Changes in kyua-cli version 0.9
===============================

Experimental version released on August 8th, 2014.

Major changes:

The internal architecture of Kyua to record the results of test suite
runs has completely changed in this release.  Kyua no longer stores all
the different test suite run results as different "actions" within the
single store.db database.  Instead, Kyua now generates a separate
results file inside ~/.kyua/store/ for every test suite run.

Due to the complexity involved in the migration process and the little
need for it, this is probably going to be the only release where the
'db-migrate' command is able to convert an old store.db file to the
new scheme.

Changes in more detail:

* Added the 'report-junit' command to generate JUnit XML result files.
  The output has been verified to work within Jenkins.

* Switched to results files specific to their corresponding test suite
  run.  The unified store.db file is now gone: 'kyua test' creates a
  new results file for every invocation under ~/.kyua/store/ and the
  'kyua report*' commands are able to locate the latest file for a
  corresponding test suite automatically.

* The 'db-migrate' command takes an old store.db file and generates one
  results file for every previously-recorded action, later deleting the
  store.db file.

* The '--action' flag has been removed from all commands that accepted
  it.  This has been superseded by the tests results files.

* The '--store' flag that many commands took has been renamed to
  '--results-file' in line with the semantical changes.

* The 'db-exec' command no longer creates an empty database when none
  is found.  This command is now intended to run only over existing
  files.


Changes in kyua-testers version 0.3
===================================

Experimental version released on August 8th, 2014.

* Made the testers set a "sanitized" value for the HOME environment
  variable where, for example, consecutive and trailing slashes have
  been cleared.  Mac OS X has a tendency to append a trailing slash to
  the value of TMPDIR, which can cause third-party tests to fail if they
  compare ${HOME} with $(pwd).

* Issues 85, 86, 90 and 92: Made the TAP parser more complete: mark test
  cases reported as TODO or SKIP as passed; handle skip plans; ignore
  lines that look like "ok" and "not ok" but aren't results; and handle
  test programs that report a pass but exit with a non-zero code.


Changes in kyua-cli version 0.8
===============================

Experimental version released on December 7th, 2013.

* Added support for Lutok 0.4.

* Issue 24: Plug the bootstrap tests back into the test suite.  Fixes
  in kyua-testers-0.2 to isolate test cases into their own sessions
  should allow these to run fine.

* Issue 74: Changed the kyuafile(5) parser to automatically discover
  existing tester interfaces.  The various *_test_program() functions
  will now exist (or not) based on tester availability, which simplifies
  the addition of new testers or the selective installation of them.


Changes in kyua-testers version 0.2
===================================

Experimental version released on December 7th, 2013.

* Issue 74: Added the kyua-tap-tester, a new backend to interact with
  test programs that comply with the Test Anything Protocol.

* Issue 69: Cope with the lack of AM_PROG_AR in configure.ac, which
  first appeared in Automake 1.11.2.  Fixes a problem in Ubuntu 10.04
  LTS, which appears stuck in 1.11.1.

* Issue 24: Improve test case isolation by confining the tests to their
  own session instead of just to their own process group.


Changes in kyua-cli version 0.7
===============================

Experimental version released on October 18th, 2013.

* Made failures from testers more resilent.  If a tester fails, the
  corresponding test case will be marked as broken instead of causing
  kyua to exit.

* Added the '--results-filter' option to the 'report-html' command and
  set its default value to skip passed results from HTML reports.  This
  is to keep these reports more succint and to avoid generating tons of
  detail files that will be, in general, useless.

* Switched to use Lutok 0.3 to gain compatibility with Lua 5.2.

* Issue 69: Cope with the lack of AM_PROG_AR in configure.ac, which
  first appeared in Automake 1.11.2.  Fixes a problem in Ubuntu 10.04
  LTS, which appears stuck in 1.11.1.


Changes in kyua-cli version 0.6
===============================

Experimental version released on February 22nd, 2013.

* Issue 36: Changed 'kyua help' to not fail when the configuration file
  is bogus.  Help should always work.

* Issue 37: Simplified the syntax() calls in configuration and Kyuafile
  files to only specify the requested version instead of also the format
  name.  The format name is implied by the file being loaded, so there
  is no use in the caller having to specify it.  The version number of
  these file formats has been bumped to 2.

* Issue 39: Added per-test-case metadata values to the HTML reports.

* Issue 40: Rewrote the documentation as manual pages and removed the
  previous GNU Info document.

* Issue 47: Started using the independent testers in the kyua-testers
  package to run the test cases.  Kyua does not implement the logic to
  invoke test cases any more, which provides for better modularity,
  extensibility and robustness.

* Issue 57: Added support to specify arbitrary metadata properties for
  test programs right from the Kyuafile.  This is to make plain test
  programs more versatile, by allowing them to specify any of the
  requirements (allowed architectures, required files, etc.) supported
  by Kyua.

* Reduced automatic screen line wrapping of messages to the 'help'
  command and the output of tables by 'db-exec'.  Wrapping any other
  messages (specially anything going to stderr) was very annoying
  because it prevented natural copy/pasting of text.

* Increased the granularity of the error codes returned by kyua(1) to
  denote different error conditions.  This avoids the overload of '1' to
  indicate both "expected" errors from specific subcommands and
  unexpected errors caused by the internals of the code.  The manual now
  correctly explain how the exit codes behave on a command basis.

* Optimized the database schema to make report generation almost
  instantaneous.

* Bumped the database schema to 2.  The database now records the
  metadata of both test programs and test cases generically, without
  knowledge of their interface.

* Added the 'db-migrate' command to provide a mechanism to upgrade a
  database with an old schema to the current schema.

* Removed the GDB build-time configuration variable.  This is now part
  of the kyua-testers package.

* Issue 31: Rewrote the Kyuafile parsing code in C++, which results in
  a much simpler implementation.  As a side-effect, this gets rid of the
  external Lua files required by 'kyua', which in turn make the tool
  self-contained.

* Added caching of various configure test results (particularly in those
  tests that need to execute a test program) so that cross-compilers can
  predefine the results of the tests without having to run the
  executables.


Changes in kyua-testers version 0.1
===================================

Experimental version released on February 19th, 2013.

This is the first public release of the kyua-testers package.

The goal of this first release is to adopt all the test case execution
code of kyua-cli 0.5 and ship it as a collection of independent tester
binaries.  The kyua-cli package will rely on these binaries to run the
tests, which provides better modularity and simplicity to the
architecture of Kyua.

The code in this package is all C as opposed to the current C++ codebase
of kyua-cli, which means that the overall build times of Kyua are now
reduced.


Changes in kyua-cli version 0.5
===============================

Experimental version released on July 10th, 2012.

* Issue 15: Added automatic stacktrace gathering of crashing test cases.
  This relies on GDB and is a best-effort operation.

* Issue 32: Added the '--build-root' option to the debug, list and test
  commands.  This allows executing test programs from a different
  directory than where the Kyuafile scripts live.  See the 'Build roots'
  section in the manual for more details.

* Issue 33: Removed the kyuaify.sh script.  This has been renamed to
  atf2kyua and moved to the kyua-atf-compat module, where it ships as a
  first-class utility (with a manual page and tests).

* Issue 34: Changed the HTML reports to include the stdout and stderr of
  every test case.

* Fixed the build when using a "build directory" and a clean source tree
  from the repository.


Changes in kyua-cli version 0.4
===============================

Experimental version released on June 6th, 2012.

* Added the 'report-html' command to generate HTML reports of the
  execution of any recorded action.

* Changed the '--output' flag of the 'report' command to only take a
  path to the target file, not its format.  Different formats are better
  supported by implementing different subcommands, as the options they
  may receive will vary from format to format.

* Added a '--with-atf' flag to the configure script to control whether
  the ATF tests get built or not.  May be useful for packaging systems
  that do not have ATF in them yet.  Disabling ATF also cuts down the
  build time of Kyua significantly, but with the obvious drawbacks.

* Grouped 'kyua' subcommands by topic both in the output of 'help' and
  in the documentation.  In general, the user needs to be aware of
  commands that rely on a current project and those commands that rely
  purely on the database to generate reports.

* Made 'help' print the descriptions of options and commands properly
  tabulated.

* Changed most informational messages to automatically wrap on screen
  boundaries.

* Rewrote the configuration file parsing module for extensibility.  This
  will allow future versions of Kyua to provide additional user-facing
  options in the configuration file.

  No syntax changes have been made, so existing configuration files
  (version 1) will continue to be parsed without problems.  There is one
  little exception though: all variables under the top-level
  'test_suites' tree must be declared as strings.

  Similarly, the '-v' and '--variable' flags to the command line must
  now carry a 'test_suites.' prefix when referencing any variables under
  such tree.


Changes in kyua-cli version 0.3
===============================

Experimental version released on February 24th, 2012.

* Made the 'test' command record the results of the executed test
  cases into a SQLite database.  As a side effect, 'test' now supports a
  '--store' option to indicate where the database lives.

* Added the 'report' command to generate plain-text reports of the
  test results stored in the database.  The interface of this command is
  certainly subject to change at this point.

* Added the 'db-exec' command to directly interact with the store
  database.

* Issue 28: Added support for the 'require.memory' test case property
  introduced in ATF 0.15.

* Renamed the user-specific configuration file from ~/.kyuarc to
  ~/.kyua/kyua.conf for consistency with other files stored in the
  ~/.kyua/ subdirectory.

* Switched to use Lutok instead of our own wrappers over the Lua C
  library.  Lutok is just what used to be our own utils::lua module, but
  is now distributed separately.

* Removed the 'Atffile's from the source tree.  Kyua is stable enough
  to generate trustworthy reports, and we do not want to give the
  impression that atf-run / atf-report are still supported.

* Enabled logging to stderr for our own test programs.  This makes it
  slightly easier to debug problems in our own code when we get a
  failing test.


Changes in kyua-cli version 0.2
===============================

Experimental version released on August 24th, 2011.

The biggest change in this release is the ability for Kyua to run test
programs implemented using different frameworks.  What this means is
that, now, a Kyua test suite can include not only ATF-based test
programs, but also "legacy" (aka plain) test programs that do not use
any framework.  I.e. if you have tests that are simple programs that
exit with 0 on success and 1 on failure, you can plug them in into a
Kyua test suite.

Other than this, there have been several user-visible changes.  The most
important are the addition of the new 'config' and 'debug' subcommands
to the 'kyua' binary.  The former can be used to inspect the runtime
configuration of Kyua after parsing, and the latter is useful to
interact with failing tests cases in order to get more data about the
failure itself.

Without further ado, here comes the itemized list of changes:

* Generalized the run-time engine to support executing test programs
  that implement different interfaces.  Test programs that use the ATF
  libraries are just a special case of this.  (Issue 18.)

* Added support to the engine to run "plain" test programs: i.e. test
  programs that do not use any framework and report their pass/fail
  status as an exit code.  This is to simplify the integration of legacy
  test programs into a test suite, and also to demonstrate that the
  run-time engine is generic enough to support different test
  interfaces.  (Issue 18.)

* Added the 'debug' subcommand.  This command allows end users to tweak
  the execution of a specific test case and to poke into the behavior of
  its execution.  At the moment, all this command allows is to view the
  stdout and stderr of the command in real time (which the 'test'
  command currently completely hides).

* Added the 'config' subcommand.  This command allows the end user to
  inspect the current configuration variables after evaluation, without
  having to read through configuration files.  (Issue 11.)

* Removed the test_suites_var function from configuration files.  This
  was used to set the value of test-suite-sepecific variables, but it
  was ugly-looking.  It is now possible to use the more natural syntax
  'test_suites.<test-suite-name>.<variable> = <value>'.  (Issue 11.)

* Added a mechanism to disable the loading of configuration files
  altogether.  Needed for testing purposes and for scriptability.
  Available by passing the '--config=none' flag.

* Enabled detection of unused parameters and variables in the code and
  fixed all warnings.  (Issue 23.)

* Changed the behavior of "developer mode".  Compiler warnings are now
  enabled unconditionally regardless of whether we are in developer mode
  or not; developer mode is now only used to perform strict warning
  checks and to enable assertions.  Additionally, developer mode is now
  only automatically enabled when building from the repository, not for
  formal releases.  (Issue 22.)

* Fixed many build and portability problems to Debian sid with GCC 4.6.3
  and Ubuntu 10.04.1 LTS.  (Issues 20, 21, 26.)


Changes in kyua-cli version 0.1
===============================

Experimental version released on June 23rd, 2011.

This is the first public release of the kyua-cli package.

The scope of this release is to provide functional replacement for the
'atf-run' utility included in the atf package.  At this point, 'kyua'
can reliably run the NetBSD 5.99.53 test suite delivering the same
results as 'atf-run'.

The reporting facilities of this release are quite limited.  There is
no replacement for 'atf-report' yet, and there is no easy way of
debugging failing test programs other than running them by hand.  These
features will mark future milestones and therefore be part of other
releases.

Be aware that this release has suffered very limited field testing.
The test suite for kyua-cli is quite comprehensive, but some bugs may
be left in any place.
