Skip to content

@BeforeMethod() does not get called before each call? #896

Description

@Ocramius

I was working on laminas/laminas-servicemanager#93 today, and noticed that we still have the codebase sprinkled with blocks referencing #304

In fact, #304 has been closed as "old", but still applies today.

My assumption was that @BeforeMethods would be called before each call to said bench method: doesn't seem to be the case, and that leads to benchmarking warmed up caches too, which is a problem (especially if we're benchmarking said warmup).

In following example, I would expect 100 calls to mySetup(), but only 10 are occurring:

/** 
 * @BeforeMethods
 * @Revs(10)
 * @Iterations(10)
 */ 
class MyBench
{
    public function mySetup(): void
    {
        reset_all_the_things_here();
    }

    public function benchSomething(): void
    {
        // irrelevant
    }
}

What's the best way forward here? Is a change in @BeforeMethods viable? It would change the benchmark results for bench suites I've worked on so far, massively, but it would lead to more honest results.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions