Skip to content

Tests write to testOutput without creating it, so they depend on test order #108

Description

@jorabin

Several tests write files into testOutput without creating that directory first. They pass only when some other test has already created it, so whether they pass depends on test order. This surfaced when the 2.x build first ran on GitHub Actions, where Issue88Test ran early and failed:

java.nio.file.NoSuchFileException: testOutput/Issue88.kdbx

Affected:

  • 2.x: Issue87Test, Issue88Test and SimpleQuickStartTest in example. QuickStart.ensureOutputDir() existed but was never called, and PropertyValueChecks and SaveAndReloadChecks each had their own copy.
  • 3.x: Issue87Test, Issue88Test, KdbxFileSaveAndReloadTest and ProtectedPropertyTest2.

Fix

Add one shared helper to TestUtil (in util for 2.x, test for 3.x) that returns a path in testOutput and creates the directory, and use it in every test that writes output, replacing the separate copies. Check each affected test passes when run on its own with no testOutput directory.

Activity

  1. jorabin commented on Oct 3, 2026

    @jorabin
    OwnerAuthor

    Fixed on develop for 2.2.7 in de24032 and on v3-develop for 3.0.1 in 2ceef13: TestUtil.testOutputPath(name) creates testOutput and returns the path, and every affected test uses it. Each was checked to pass when run on its own with no testOutput directory.

    A correction to the description: in 3.x, KdbxFileSaveAndReloadTest and ProtectedPropertyTest2 didn't actually fail, because every class that uses them extends DatabaseTestBase, which creates the directory. They relied on that silently, so they use the helper now too. Only Issue87Test and Issue88Test were order-dependent in 3.x.

    QuickStart in 2.x also wrote migration.kdbx without creating the directory; it now creates it.

  2. jorabin commented on Oct 7, 2026

    @jorabin
    OwnerAuthor

    This continues #83: PR #85, which fixed #83, also included 51949dc ("Make sure 'testOutput' directory exists before accessing it"), but only for QuickStart. This issue does the same for every test that writes to testOutput.

    • v3: 2ceef13, released in 3.1.0 (the 3.0.1-SNAPSHOT it was labelled for became 3.1.0).
    • 2.x: de24032 on develop, to be released in 2.2.7; this issue stays open until then.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions