resilience4j

Resilience4j is a fault tolerance library for Java™

Resilience4j is a lightweight fault tolerance library inspired by Netflix Hystrix, but designed for functional programming. Lightweight, because the library only uses Vavr, which does not have any other external library dependencies. Netflix Hystrix, in contrast, has a compile dependency to Archaius which has many more external library dependencies such as Guava and Apache Commons Configuration.

Resilience4j provides higher-order functions (decorators) to enhance any functional interface, lambda expression or method reference with a Circuit Breaker, Rate Limiter, Retry or Bulkhead. You can stack more than one decorator on any functional interface, lambda expression or method reference. The advantage is that you have the choice to select the decorators you need and nothing else.

Supplier<String> supplier = () -> backendService.doSomething(param1, param2);

Supplier<String> decoratedSupplier = Decorators.ofSupplier(supplier)
  .withRetry(Retry.ofDefaults("name"))
  .withCircuitBreaker(CircuitBreaker.ofDefaults("name"))
  .withBulkhead(Bulkhead.ofDefaults("name"));  

String result = Try.ofSupplier(decoratedSupplier)
  .recover(throwable -> "Hello from Recovery").get();

With Resilience4j you don’t have to go all-in, you can pick what you need.

Get Started

RateLimiter

Getting started with resilience4j-ratelimiter

Introduction

Rate limiting is an imperative technique to prepare your API for scale and establish high availability and reliability of your service. But also, this technique comes with a whole bunch of different options of how to handle a detected limits surplus, or what type of requests you want to limit. You can simply decline this over limit request, or build a queue to execute them later or combine these two approaches in some way.

Internals

Resilience4j provides a RateLimiter which splits all nanoseconds from the start of epoch into cycles. Each cycle has a duration configured by RateLimiterConfig.limitRefreshPeriod. At the start of each cycle, the RateLimiter sets the number of active permissions to RateLimiterConfig.limitForPeriod.
For the RateLimiter callers it really looks like this, but for the AtomicRateLimiter implementation has some optimizations under the hood that will skip this refresh, if RateLimiter is not used actively.

The default implementation of RateLimiter is AtomicRateLimiter which manages it's state via AtomicReference. The AtomicRateLimiter.State is completely immutable and has the folowing fields:

  • activeCycle - cycle number that was used by the last call
  • activePermissions - count of available permissions after the last call.
    Can be negative if some permissions where reserved
  • nanosToWait - count of nanoseconds to wait for permission for the last call

There is also a SemaphoreBasedRateLimiter which uses Semaphores and a scheduler that will refresh permissions after each RateLimiterConfig#limitRefreshPeriod.

Create a RateLimiterRegistry

Just like the CircuitBreaker module, this module provides an in-memory 'RateLimiterRegistry' which you can use to manage (create and retrieve) RateLimiter instances.

RateLimiterRegistry rateLimiterRegistry = RateLimiterRegistry.ofDefaults();

Create and configure a RateLimiter

You can provide a custom global RateLimiterConfig. In order to create a custom global RateLimiterConfig, you can use the RateLimiterConfig builder. You can use the builder to configure the following properties.

Config property
Default value
Description

timeoutDuration

5 [s]

The default wait time a thread waits for a permission

limitRefreshPeriod

500 [ns]

The period of a limit refresh. After each period the rate limiter sets its permissions count back to the limitForPeriod value

limitForPeriod

50

The number of permissions available during one limit refresh period

For example you want to restrict the calling rate of some method to be not higher than 10 req/ms.

RateLimiterConfig config = RateLimiterConfig.custom()
  .limitRefreshPeriod(Duration.ofMillis(1))
  .limitForPeriod(10)
  .timeoutDuration(Duration.ofMillis(25))
  .build();

// Create registry
RateLimiterRegistry rateLimiterRegistry = RateLimiterRegistry.of(config);

// Use registry
RateLimiter rateLimiterWithDefaultConfig = rateLimiterRegistry
  .rateLimiter("name1");

RateLimiter rateLimiterWithCustomConfig = rateLimiterRegistry
  .rateLimiter("name2", config);

Decorate and execute a functional interface

As you can guess RateLimiter has all sort of higher order decorator functions just like CircuitBreaker. You can decorate any Callable, Supplier, Runnable, Consumer, CheckedRunnable, CheckedSupplier, CheckedConsumer or CompletionStage with a RateLimiter .

// Decorate your call to BackendService.doSomething()
CheckedRunnable restrictedCall = RateLimiter
    .decorateCheckedRunnable(rateLimiter, backendService::doSomething);

Try.run(restrictedCall)
    .andThenTry(restrictedCall)
    .onFailure((RequestNotPermitted throwable) -> LOG.info("Wait before call it again :)"));

You can use changeTimeoutDuration and changeLimitForPeriod methods to change rate limiter params in runtime.
New timeout duration won’t affect threads that are currently waiting for permission.
New limit won’t affect current period permissions and will apply only from next one.

// Decorate your call to BackendService.doSomething()
CheckedRunnable restrictedCall = RateLimiter
    .decorateCheckedRunnable(rateLimiter, backendService::doSomething);

// durring second refresh cycle limiter will get 100 permissions
rateLimiter.changeLimitForPeriod(100);

Consume emitted RegistryEvents

You can register event consumer on a RateLimiterRegistry and take actions whenever a RateLimiter is created, replaced or deleted.

RateLimiterRegistry registry = RateLimiterRegistry.ofDefaults();
registry.getEventPublisher()
  .onEntryAdded(entryAddedEvent -> {
    RateLimiter addedRateLimiter = entryAddedEvent.getAddedEntry();
    LOG.info("RateLimiter {} added", addedRateLimiter.getName());
  })
  .onEntryRemoved(entryRemovedEvent -> {
    RateLimiter removedRateLimiter = entryRemovedEvent.getRemovedEntry();
    LOG.info("RateLimiter {} removed", removedRateLimiter.getName());
  });

Consume emitted RateLimiterEvents

The RateLimiter emits a stream of RateLimiterEvents. An event can be a successful permission acquire or acquire failure.
All events contains additional information like event creation time and rate limiter name.
If you want to consume events, you have to register an event consumer.

rateLimiter.getEventPublisher()
    .onSuccess(event -> logger.info(...))
    .onFailure(event -> logger.info(...));

You can use RxJava or RxJava2 Adapters to convert the EventPublisher into a Reactive Stream.

ReactorAdapter.toFlux(rateLimiter.getEventPublisher())
    .filter(event -> event.getEventType() == FAILED_ACQUIRE)
    .subscribe(event -> logger.info(...))

What's Next

Retry

RateLimiter


Getting started with resilience4j-ratelimiter

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.