Microservices interaction at scale using Apache Kafka

Intro Why do people do microservices architecture? Let’s start with a review of a classical, old-school, n-tier application, which is often called a monolith: Everyone worked with this types of applications, we were studying them in university, we did lot’s of projects with this architecture and at first glance, it’s a good architecture. We can immediately start doing systems on this architecture, they are perfectly simple and easy to implement. But as such system becomes bigger, it starts receiving lots of different issues. The first issue is about scalability, it doesn’t scale well. If some part of such system will need to scale, then the overall system should be scaled. Since the system is divided into modules, it runs as one physical process and there’s no option to scale one module. This can lead to expensive large instances, which you will have to maintain. The second issue is about engineers. Since it will become enormously big, you will have to hire a really big team for such system. Evolution How can you address this issues? One of the ways you can evolve is to make a transition to a microservices architecture The way it works is that you have a...

Concurrency structures written using core java

Motivation The reason for writing this article is a lack of my understanding of Java concurrency. That’s why I decided to create a post with some structures, which I wrote by myself. For sure, you may agree or disagree with me, I encourage to post your thoughts in comments so we can discuss them there. List of structures Implemented: Publish-Subscribe Queue Will implement them later: Fixed Thread Pool Cached Thread Pool CountDownLatch CyclicBarrier Phaser Semaphore Exchanger Rate Limiter Lock, ReentrantLock, ReentrantReadWriteLock ConcurrentHashMap AtomicInteger(For fun) Publish/Subscribe What is Publish-Subscribe, in a nutshell? Obviously, there someone, who has something share something to others, and want to be sure, that everyone will receive this message. The idea of following structure is to have a publish-subscribe mechanism, which will act asynchronously, without usage of high-level classes from java.util.concurrent package: Consumer - entity, which will consume messages from main thread. PubSubModel - main thread, which will send messages to the consumers. Formally, you can treat it as a consumer. Code can be found here. Let’s go over this code and try all the details: First of all, there will be one main thread, which will read lines from console and act as a producer, so...

Database Migration tools for Mongo DB

When it comes to the problem of migrating database structure, some of you may think about relational databases: there is a strict schema, and to remove something(field, table, index, etc.), you need to take action: execute an SQL statement. But when you work on schema-less databases, it may look like you don’t need those migrations. But to be honest, are schema-less database are schemaless? In fact, you have more freedom in column and document-based database, but sooner or later you will have to modify some of the results of your work: remove the index, transform column format, etc. That’s why with the help of this article I would like to review the available tools for Mongo migrations. Mongobee If you use Spring in your project, then MongoBee should be the most suitable tool for you. The idea is that you write Java methods(changesets), which describe what changes need to be done. The method annotated by @ChangeSet is taken and applied to the database. Mongobee stores changesets history in dbchangelog collection. If you are a Spring guy, and like Java config among others, then you should choose this tool. You have two options how to run Mongobee - inside Spring container...

How to map Spring Boot properties to Java class

Have you ever had a need to use some values from application.properties or **application.yml? How did you take them out? Personally, I always used @Value annotation: @Value("${graphite.host}") private String graphiteHost; It wasn’t the best way to work with my properties. However, I didn’t know a better approach. Then, I found @ConfigurationProperties - annotation from Boot package, which has everything you need to map your properties. Given Let’s say, your application.yml looks like following: graphite: enabled: true host: localhost port: 2003 amountOfTimeBetweenPolls: 20000 When You need to create a bunch of classes, which you will be autowiring in all parts of your code. I’m using Project Lombok for skipping java formalities, if you are not, then create getters and setters for your classes. package org.rngr.properties; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import javax.validation.constraints.NotNull; @ConfigurationProperties(ignoreUnknownFields = true) @Data public class ApplicationProperties { @NotNull private GraphiteProperties graphite; } Pay attention to @ConfigurationProperties annotation, it’s playing a key role here. Don’t forget about subclass: package org.rngr.properties; import lombok.Data; @Data public class GraphiteProperties { private boolean enabled; private String host; private int port; private int amountOfTimeBetweenPolls; } In the end, you need to enable configuration properties: package org.rngr; import lombok.extern.slf4j.Slf4j; import org.rngr.config.*; import org.rngr.properties.ApplicationProperties; import org.springframework.beans.factory.annotation.Autowired; import...

How to trace your logs using SLF4J MDC

How do you use your logs for searching problem requests? For instance, you got a problem response, with all headers, response body, and you need to find appropriate logs. How would you do that? Personally, I found it useful to write some words about MDC - Mapped Diagnostic Context. Shortly, it is a concept of mapping request specific information. Usage We will configure MDC in Spring Boot application. We will use SLF4J on top of Logback implementation. Using it all together, we will create a unique requestId for each request in our system. Components We will use 4 components here: Spring Boot, Slf4j, Logback and Java Spring Boot Spring Boot will be used for managing dependency injection and registering pure Java Filter. Slf4j Simple Logging Facade is used for following abstraction principles. Additionally, MDC class is located inside slf4j dependency. Similar classes are inside log4j and logback dependencies. Logback Logback is used as one of logging providers Pure Java Java is used for writing simple Java Filter. Affected files MDCFilter package org.startup.queue.filter; import org.slf4j.MDC; import org.springframework.stereotype.Component; import javax.servlet.*; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.IOException; import java.util.UUID; @Component public class MDCFilter implements Filter { @Override public void init(FilterConfig filterConfig) throws ServletException...