The Case of the Sometimes-Null Constant

Published: January 31, 2026

#oss #debugging

This is the story of a flaky test that was passing for all the wrong reasons.

It's about null comparisons, circular dependencies, class initialization order, and how the JVM can surprise you when you least expect it.

The Crime Scene

I came across an issue in the Micrometer metrics library about flaky tests in NoopObservationRegistryTests.

The tests were randomly failing with a cryptic assertion error:

java.lang.AssertionError:
Expecting actual:
  null
and:
  io.micrometer.observation.NoopObservation$NoopScope@2f08c4b
to refer to the same object

The tests were using AssertJ's isSameAs() to verify that certain fields were pointing to the same singleton instance. Most times they passed. But every once in a while they failed. Classic flaky test behavior.

"It must be a concurrency issue," I thought. "Or maybe some shared state between tests?"

But as I dug deeper, I realized something far more interesting was happening.

Finding a Lead

Here's a simplified version of what the test was doing:

@Test
void shouldNotRespectScopesIfNoopRegistryIsUsed() {
    ObservationRegistry registry = ObservationRegistry.NOOP;
    Observation observation = Observation.start("foo", registry);

    try (Observation.Scope scope = observation.openScope()) {
        then(registry.getCurrentObservationScope())
            .isSameAs(Observation.Scope.NOOP);
    }
}

The assertion was checking that the current scope was the same instance as Observation.Scope.NOOP —a public static final singleton.

I set a breakpoint right before the assertion in one of the passing test runs. Both the actual and expected values were null —that's why the tests were passing most of the time!

The test was effectively doing:

then(null).isSameAs(null);

When both were null, the test passed. But sometimes, very rarely, the Observation.Scope.NOOP wasn't null —and the test failed.

So, getCurrentObservationScope() was always returning null. The flaky test wasn't revealing a bug in the test logic. It was revealing a bug in the production code.

That's good to know, it can be dealt with in a different issue. But, for me, the more interesting question was:

How could Scope.NOOP be null as well? It's a singleton static final field - is this even possible?

Following the Trail

Let's take a closer look at the simplified code snippet:

interface Scope {
    Scope NOOP = NoopScope.INSTANCE;
    // ...
}

class NoopScope implements Scope {
    static final Scope INSTANCE = new NoopScope();
    // ...
}

Do you notice anything peculiar? Take a moment to think about it.

When the JVM initializes NoopScope.INSTANCE first:

• It loads the NoopScope class
• Since NoopScope implements Scope, it loads the Scope interface
• The interface tries to initialize Scope.NOOP by referencing NoopScope.INSTANCE
• But NoopScope.INSTANCE isn't fully initialized yet—it's still being created!
• Scope.NOOP ends up as null

The initialization order depended on which class the JVM happened to load first. Hence, the flaky behavior.

The Smoking Gun

I was very surprised by this behavior, and I needed to be absolutely sure.

Turns out even a plain-old System.out.println() would be enough to change the class initialization order, and reproduce the issue reliably:

@Test
void test() {
    // this sysout will force initializing NoopScope.INSTANCE first
    System.out.println(NoopScope.INSTANCE);

    then(Scope.NOOP).isNotNull(); // FAILS
}

Of course, we'll run this test in isolation to avoid having other tests interfere with the order of loading the classes.

Bingo! By forcing the classloader to initialize NoopScope first, we could reliably make Scope.NOOP become null every single time.

Comment out that first println() and run again —the behavior changes completely! The test passes.

The Solution

I tried several approaches to fix this, but they all broke backward compatibility. Needless to say, that was not an option for a widely-used library such as Micrometer.

The public API couldn't change—Scope.NOOP needed to remain a public static final field.

That's when the Micrometer team came up with an ingenious solution: replace the concrete class references with anonymous inner classes.

Of course, the full implementation is more complex, feel free to check out all the changes here. But, for now, let's stick to our simplified code sample.

Instead of:

interface Scope {
    Scope NOOP = NoopScope.INSTANCE;  // Circular reference!
}

They used:

interface Scope {
    // Anonymous class - no circular reference possible
    Scope NOOP = new Scope() {
        @Override
        public Observation getCurrentObservation() {
            // ... implementation ...
        }
        // ... other methods...
    };
}

The anonymous inner class is part of the Scope interface's initialization, so there's no external dependency that could cause a circular initialization.

Simply put, the Scope interface still has a public static Scope field, but we removed the references to any concrete implementation.

How to Catch This Early

Apart from fixing the bug, we need to make sure this kind of issue wouldn't happen again in the future.

Turns out there's an ErrorProne check specifically for this: ClassInitializationDeadlock. This static analysis rule can detect circular initialization dependencies at compile time.

Adding it to the build configuration:

errorprone {
    check("ClassInitializationDeadlock", CheckSeverity.ERROR)
}

Now the build would fail immediately if anyone introduced this pattern again.

In a different PR, this was integrated, too, into Micrometer's build process.

Lessons Learned

• Flaky tests are sometimes symptoms of real bugs
• Circular dependencies between static fields can create subtle, non-deterministic bugs
• ErrorProne and other static analysis tools can catch these issues before they reach production

You can see the full investigation and discussion in the original issue.