Detecting memory leaks using JVisualVM and Memory Analyzer Tool

Few days ago I had a problem on one of the projects that I am working on, we had a memory leak. During the two days period our services crashed three times, so I decided to investigate it. Everything which I’m going to talk about is not a rocket science, there’s no clever and tricky tips, it’s just a straighforward explanation how you can find memory leaks. Exposing JMX I had a problem on a production instance, so I started my services with JMX feature enabled. Just start your apps with following params: -Djavax.management.builder.initial= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=${whatever_port} -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false Starting JVisualVM Just enter your terminal and type jvisualvm. You should get following screen: Add a remote connection, specify JMX port and connect. Waiting You have to wait some time before retained memory will take place and you will be able to analyse it. It’s up to you how long to wait, in my case, it was enough to wait 4-5 hours to get 100% proof of what part of the system is leaking. Getting heap dump Now go to Monitor section, press Heap Dump button and specify path where heap dump should be saved. In my case it was /tmp/**.hproof. Then...

How we wrote chicken egg counter on a Raspberry PI

How it started Besides my main work on Upwork I quite often pick different projets. So I found a project, where I had to write a program for recognizing chicken eggs on a factory stream line. Customer wanted to install the application on computer with web camera, put this camera at a top of stream line and the application had to calculate eggs and send them to the DB. He also wanted to run this program on a cheap computer. The quality of the network in the factory isn’t stable, so the program had to be resilient to outstand the network issues. There was enough challenges for me, so I decided to participate on this project. The biggest challenge here was that I had no serious experience with OpenCV and image recognition, so I wanted to test myself if I can deep dive into unknown field and return with successful result. Customer wanted to have 99% of recognition. This whole post will be a story how this application was designed, how it was written and what problems did I faced during the development. I will try to explain each architecture decision, from the beginning and to the end of the...

Migrating from Ghost to Jekyll

Jekyll Why Jekyll ? I decided to use Jekyll, because I had a blog on Ghost platform. I was waiting for a new 1.0 release, but then I suddenly realize that I don’t want to use it, because: I have to maintain it on my own I have to pay 10$ every week for 1GB DigitalOcean instance SSL certificates Ghost is written on javascript, so there’s a specific scalability Jekyll, on the other hand, is hosted on GitHub and is a great and modern instrument for writing your blog. The idea is that you store all your images and posts on GitHub. Speed My first question was about performance. If it’s a static files, hosted on GitHub, then they have to be extremely slow. No, that’s not true and according to my measures, new version on Jekyll is even faster, than Ghost version. Convenience Another question I asked myself was how I will write posts. Because Jekyll has no admin gui, I need to find a way to write posts. I find MacDown tool very convenient for writing posts on my local laptop. Another problem comes about pre-showing posts. For example, you want to see how your post will look...

Microservices interaction at scale using Apache Kafka

Intro Why do people do microservices architecture? Let’s start with a review of a classical, old-school, n-tier application, which is often called a monolith: Everyone worked with this types of applications, we were studying them in university, we did lot’s of projects with this architecture and at first glance, it’s a good architecture. We can immediately start doing systems on this architecture, they are perfectly simple and easy to implement. But as such system becomes bigger, it starts receiving lots of different issues. The first issue is about scalability, it doesn’t scale well. If some part of such system will need to scale, then the overall system should be scaled. Since the system is divided into modules, it runs as one physical process and there’s no option to scale one module. This can lead to expensive large instances, which you will have to maintain. The second issue is about engineers. Since it will become enormously big, you will have to hire a really big team for such system. Evolution How can you address this issues? One of the ways you can evolve is to make a transition to a microservices architecture The way it works is that you have a...

Concurrency structures written using core java

Motivation The reason for writing this article is a lack of my understanding of Java concurrency. That’s why I decided to create a post with some structures, which I wrote by myself. For sure, you may agree or disagree with me, I encourage to post your thoughts in comments so we can discuss them there. List of structures Implemented: Publish-Subscribe Queue Will implement them later: Fixed Thread Pool Cached Thread Pool CountDownLatch CyclicBarrier Phaser Semaphore Exchanger Rate Limiter Lock, ReentrantLock, ReentrantReadWriteLock ConcurrentHashMap AtomicInteger(For fun) Publish/Subscribe What is Publish-Subscribe, in a nutshell? Obviously, there someone, who has something share something to others, and want to be sure, that everyone will receive this message. The idea of following structure is to have a publish-subscribe mechanism, which will act asynchronously, without usage of high-level classes from java.util.concurrent package: Consumer - entity, which will consume messages from main thread. PubSubModel - main thread, which will send messages to the consumers. Formally, you can treat it as a consumer. Code can be found here. Let’s go over this code and try all the details: First of all, there will be one main thread, which will read lines from console and act as a producer, so...