External services or APIs may have usage limits or they just cannot handle loads of requests without failing. This post explains how to create a Spring Framework based aspect that can be used to throttle any adviced method calls with Guava’s rate limiter. The following implementation requires Java 8, Spring AOP and Guava.

Let’s start with an annotation that is used to advice any Spring AOP enabled method call.

The annotation defines two things: the rate limit as in queries (or permits) per second and an optional key to identify a rate limiter. Multiple methods can use the same rate limiter if the keys are equal. For example when an API is called with different parameters from different methods the desired total queries per second will not exceed.

Next thing is the actual throttling aspect which is implemented as a Spring Framework component. It is fairly simple to use the aspect in any context, with or without Spring Framework.

The class defines an additional interface and a default implementation for a key factory that is used if the annotation does not provide an explicit key for a rate limiter. The key factory can use the join point (basically a method call) and the provided annotation to create a suitable key for the rate limiter. The aspect also uses concurrent hashmap to store the rate limiter instances. The aspect is defined as a singleton but the rateLimit method can be called from multiple threads so the concurrent hashmap ensures we allocate only single rate limiter per unique key. Constructor injection in the aspect utilizes the optional injection support of Spring Framework. If there is no KeyFactory bean defined in the context, the default key factory is used.

The class is annotated with @Aspect and @Component so that Spring understands an aspect is defined and enables the @Before advice. @Before advice contains only one pointcut which requires a RateLimit annotation and binds it to the limit parameter of the method. The throttling implementation is quite simple. First a key is created for the rate limiter. Then the key is used to find or create a limiter and finally the limiter is acquired for a permission.

There’s a small gotcha in the rate limiter key creation. The key defined by the annotation is converted to an optional, but optional’s orElse method cannot be used due to performance reasons. Optional’s orElse method takes a value which we need to create in any case, when the optional is present and when it’s not. The other method orElseGet on the other hand takes a supplier which allows lazy evaluation of the value only when the optional is not present. The key factory’s createKey may be an expensive operation so the supplier version is used.

Concurrent hashmap contains a handy method computeIfAbsent that atomically finds or creates a value based on a key and a defined function. This allows simple and concise lazy initialization of the map values. The rate limiters are created on demand and guaranteed to have only single instance per unique limiter key.

The default key factory implementation uses a helper method from JoinPointToStringHelper that converts a join point to textual representation.

Finally the throttling can be applied to any Spring enabled method by just adding the @RateLimit annotation.

One might wonder if this solution scales out very well? No, it really doesn’t. Guava’s rate limiter blocks the current thread so if there’s a burst of asynchronous calls against the throttled service lots of threads will be blocked and might result exhaust of free threads. Another issue raises if the services are replicated in multiple applications or JVM instances. There is no global synchronization of a limiter rate. This implementation works fine for single application living in single JVM with decent load to throttled methods.

Further reading:

Coveralls is web service created by LEMUR Heavy that helps tracking code coverage over time. Coveralls is free to use for open source projects and it requires that the projects are hosted at GitHub. The service was originally created for Ruby projects, but the provided API allows anybody to create code coverage report clients. Numerous integrations have been created for different languages such as Python, PHP, Node.js and Scala by the open source community, but there was no support for Java projects. I find this odd, because in the enterprise Java world such metrics are everyday life. There are plenty of different coverage tools and metrics platforms available like Cobertura, JaCoCo, SonarQube (previously known as Sonar) and so on. Continue reading

Recently an encoding issue of HTTP form POST requests with Tomcat was found in a project I work. The form submit was triggered with Javascript and the normal submit behavior was disabled. The form page was rendered with Content-Type: text/html; charset=UTF-8 header, the HTML head section had <meta charset="utf-8"> hint and the form was defined as normal <form> without any additional attributes. This should have resulted the form to posted with application/x-www-form-urlencoded content type using UTF-8 encoding. But no, this did not work with Tomcat, for unknown reason, though everything worked fine with Jetty application server. Continue reading

Java Comparable interface provides a way to do natural ordering for classes implementing the interface. Natural ordering makes sense for scalars and other quite simple objects, but when we come to more business oriented domain objects the natural ordering becomes much more complicated. A transaction object’s natural ordering from business manager’s point of view could be the value of the transaction, but from the system admin’s point of view the natural ordering could be the speed of the transaction. In most cases, there is no clear natural ordering for business domain objects. Continue reading