CircuitBreaker

Introduction

The CircuitBreaker is implemented via a finite state machine with three normal states: CLOSED, OPEN and HALF_OPEN and two special states DISABLED and FORCED_OPEN.

The CircuitBreaker uses a Ring Bit Buffer in the CLOSED state to store the success or failure statuses of function calls. A successful function call is stored as a 0 bit and a failed call is stored as a 1 bit. The Ring Bit Buffer has a configurable fixed-size. The Ring Bit Buffer uses internally a BitSet-like data structure to store the bits which is saving memory compared to a boolean array. The BitSet uses a long[] array to store the bits. That means the BitSet only needs an array of 16 long (64-bit) values to store the status of 1024 calls.

The following diagram shows what a Ring Buffer would look like for only 12 results:

The Ring Bit Buffer must be full, before the failure rate can be calculated. For example, if the size of the Ring Buffer is 10, then at least 10 calls must evaluated, before the failure rate can be calculated. If only 9 calls have been evaluated the CircuitBreaker will not trip open even if all 9 calls have failed.

The state of the CircuitBreaker changes from CLOSED to OPEN when the failure rate is above a configurable threshold. Then all calls are rejected for a configurable time duration. The CircuitBreaker throws a CallNotPermittedException when it is OPEN.

After the time duration has elapsed, the CircuitBreaker state changes from OPEN to HALF_OPEN and allows a configurable number of calls to see if the backend is still unavailable or has become available again. The CircuitBreaker uses another configurable Ring Bit Buffer to evaluate the failure rate in the HALF_OPEN state. If the failure rate is above the configured threshold, the state changes back to OPEN. If the failure rate is below or equal to the threshold, the state changes back to CLOSED.

The Circuit Breaker supports two more special states, DISABLED (always allow access) and FORCED_OPEN (always deny access). In these two states no Circuit Breaker events (apart from the state transition) are generated, and no metrics are recorded. The only way to exit from those states are to trigger a state transition or to reset the Circuit Breaker.

Tthe CircuitBreaker is thread-safe as follows :

The state of a CircuitBreaker is stored in a AtomicReference
The CircuitBreaker uses atomic operations to update the state with side-effect-free functions.
Updates to the Ring Bit Buffer are synchronized

That means atomicity should be guaranteed and only one thread is able to update the state or the Ring Bit Buffer at a point in time.

But the CircuitBreaker does not synchronize the function call. That means the function call itself is not part of the critical section.
Otherwise a CircuitBreaker would introduce a huge performance penalty and bottleneck. A slow function call would have a huge negative impact to the overall performance/throughput.

If 20 concurrent threads ask for the permission to execute a function and the state of the CircuitBreaker is closed, all threads are allowed to invoke the function. Even if the Ring Bit Buffer size is 15. The size of the Ring Bit Buffer does not mean that only 15 calls are allowed to run concurrently. If you want to restrict the number of concurrent threads, please use a Bulkhead. You can combine a Bulkhead and a CircuitBreaker.

Example with 1 Thread:

Example with 3 Threads:

Create a CircuitBreakerRegistry

Resilience4j comes with an in-memory CircuitBreakerRegistry based on a ConcurrentHashMap which provides thread safety and atomicity guarantees. You can use the CircuitBreakerRegistry to manage (create and retrieve) CircuitBreaker instances. You can create a CircuitBreakerRegistry with a global default CircuitBreakerConfig for all of your CircuitBreaker instances as follows.

CircuitBreakerRegistry circuitBreakerRegistry = 
  CircuitBreakerRegistry.ofDefaults();

Create and configure a CircuitBreaker

You can provide your own custom global CircuitBreakerConfig. In order to create a custom global CircuitBreakerConfig, you can use the CircuitBreakerConfig builder. You can use the builder to configure the following properties.

Config property	Default Value	Description
failureRateThreshold	50	The failure rate threshold in percentage above which the CircuitBreaker should trip open and start short-circuiting calls
ringBufferSizeInHalfOpenState	10	The size of the ring buffer when the CircuitBreaker is half-open. This ring buffer is used when the breaker transitions from open to half-open to decide whether the circuit is healthy or not.
ringBufferSizeInClosedState	100	The size of the ring buffer when the CircuitBreaker is closed.The ring buffer needs to be filled before the failure rate is calculated.
waitDurationInOpenState	60 [s]	The time that the CircuitBreaker should wait before transitioning from open to half-open.
automaticTransition FromOpenToHalfOpenEnabled	false	If set to true it means that the CircuitBreaker will automatically transition from open to half-open state and not call is need to trigger the transition.
recordExceptions	empty	A list of exceptions which should count as a failure
ignoreExceptions	empty	A list of exceptions which should be ignored and not count as a failure.
recordFailure	throwable -> true	A custom Predicate which evaluates if an exception should be recorded as a failure. By default all exceptions are recored as failures.

// Create a custom configuration for a CircuitBreaker
CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()
  .failureRateThreshold(50)
  .waitDurationInOpenState(Duration.ofMillis(1000))
  .ringBufferSizeInHalfOpenState(2)
  .ringBufferSizeInClosedState(2)
  .recordFailure(e -> INTERNAL_SERVER_ERROR
                 .equals(getResponse().getStatus()))
  .recordExceptions(IOException.class, TimeoutException.class)
  .ignoreExceptions(BusinessException.class, OtherBusinessException.class)
  .build();

// Create a CircuitBreakerRegistry with a custom global configuration
CircuitBreakerRegistry circuitBreakerRegistry 
  CircuitBreakerRegistry.of(circuitBreakerConfig);

// Get or create a CircuitBreaker from the CircuitBreakerRegistry 
// with the global default configuration
CircuitBreaker bulkheadWithDefaultConfig = 
  circuitBreakerRegistry.circuitBreaker("name1");

// Get or create a CircuitBreaker from the CircuitBreakerRegistry 
// with a custom configuration
CircuitBreaker bulkheadWithCustomConfig = circuitBreakerRegistry
  .circuitBreaker("name2", circuitBreakerConfig);

You can add configurations which can be shared by multiple CircuitBreaker instances.

CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()
  .failureRateThreshold(70)
  .build();

circuitBreakerRegistry.addConfiguration("someSharedConfig", config);

CircuitBreaker circuitBreaker = circuitBreakerRegistry
  .circuitBreaker("name", "someSharedConfig");

You can overwrite configurations.

CircuitBreakerConfig defaultConfig = circuitBreakerRegistry
   .getDefaultConfig();

CircuitBreakerConfig overwrittenConfig = CircuitBreakerConfig
  .from(defaultConfig)
  .waitDurationInOpenState(Duration.ofSeconds(20))
  .build();

If you don’t want to use the CircuitBreakerRegistry to manage CircuitBreaker instances, you can also create instances directly.

// Create a custom configuration for a CircuitBreaker
CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom()
  .recordExceptions(IOException.class, TimeoutException.class)
  .ignoreExceptions(BusinessException.class, OtherBusinessException.class)
  .build();

CircuitBreaker customCircuitBreaker = CircuitBreaker
  .of("testName", circuitBreakerConfig);

Decorate and execute a functional interface

You can decorate any Callable, Supplier, Runnable, Consumer, CheckedRunnable, CheckedSupplier, CheckedConsumer or CompletionStage with a CircuitBreaker.
You can invoke the decorated function with Try.of(…) or Try.run(…) from Vavr. This allows to chain further functions with map, flatMap, filter, recover or andThen. The chained functions are only invoked, if the CircuitBreaker is CLOSED or HALF_OPEN.
In the following example, Try.of(…) returns a Success<String> Monad, if the invocation of the function is successful. If the function throws an exception, a Failure<Throwable> Monad is returned and map is not invoked.

// Given
CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("testName");

// When I decorate my function
CheckedFunction0<String> decoratedSupplier = CircuitBreaker
        .decorateCheckedSupplier(circuitBreaker, () -> "This can be any method which returns: 'Hello");

// and chain an other function with map
Try<String> result = Try.of(decoratedSupplier)
                .map(value -> value + " world'");

// Then the Try Monad returns a Success<String>, if all functions ran successfully.
assertThat(result.isSuccess()).isTrue();
assertThat(result.get()).isEqualTo("This can be any method which returns: 'Hello world'");

Consume emitted RegistryEvents

You can register event consumer on a CircuitBreakerRegistry and take actions whenever a CircuitBreaker is created, replaced or deleted.

CircuitBreakerRegistry circuitBreakerRegistry = CircuitBreakerRegistry.ofDefaults();
circuitBreakerRegistry.getEventPublisher()
  .onEntryAdded(entryAddedEvent -> {
    CircuitBreaker addedCircuitBreaker = entryAddedEvent.getAddedEntry();
    LOG.info("CircuitBreaker {} added", addedCircuitBreaker.getName());
  })
  .onEntryRemoved(entryRemovedEvent -> {
    CircuitBreaker removedCircuitBreaker = entryRemovedEvent.getRemovedEntry();
    LOG.info("CircuitBreaker {} removed", removedCircuitBreaker.getName());
  });

Consume emitted CircuitBreakerEvents

A CircuitBreakerEvent can be a state transition, a circuit breaker reset, a successful call, a recorded error or an ignored error. All events contains additional information like event creation time and processing duration of the call. If you want to consume events, you have to register an event consumer.

circuitBreaker.getEventPublisher()
    .onSuccess(event -> logger.info(...))
    .onError(event -> logger.info(...))
    .onIgnoredError(event -> logger.info(...))
    .onReset(event -> logger.info(...))
    .onStateTransition(event -> logger.info(...));
// Or if you want to register a consumer listening
// to all events, you can do:
circuitBreaker.getEventPublisher()
    .onEvent(event -> logger.info(...));

You could use the CircularEventConsumer to store events in a circular buffer with a fixed capacity.

CircularEventConsumer<CircuitBreakerEvent> ringBuffer = 
  new CircularEventConsumer<>(10);
circuitBreaker.getEventPublisher().onEvent(ringBuffer);
List<CircuitBreakerEvent> bufferedEvents = ringBuffer.getBufferedEvents()

You can use RxJava or RxJava2 Adapters to convert the EventPublisher into a Reactive Stream.