Always stop unused Akka actors

Always stop unused Akka actors

Recently I came across an easy but interesting problem. An Akka-based application was failing because of an OutOfMemoryError. It was a typical memory leak. Now I know that in the failing service there is only one class retaining data in a field.

The class is an Akka actor which works in the following way:

  • a new instance of the actor is created when a file needs to be downloaded and the actor receives a message
  • the actor stores the reference to the sender
  • it sends a HTTP request to a service, responses are delivered as messages to the same actor
  • the actor can receive either the whole response or a chunk. If it has received a chunk the actor keeps storing the data in a byte array until the whole file is delivered
  • finally, when the actor has the whole response, it sends the response to the original caller (using the reference stored in step 2)

Do you see the problem? It can handle only one request and it is useless after returning the content of the file. Additionally, it keeps the file in memory! There are no references to the actor in the application code, but there still is a parent-child relationship defined in the actor system. The actor still exists!

Are you interested in data engineering?

Check out my other blog https://easydata.engineering

How can I remove an actor?

Is stopping an actor sufficient to release memory? As I already mentioned I do not keep any references to the actor besides the ones already existing within the actor system. Does the actor system remove them when I stop an actor?

I was trying to find such information in the documentation and Stackoverflow. I failed. I could read Akka code or quickly test it…

How can I verify it?

What if I created an extremely simple application which uses Akka to waste memory?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import akka.actor.{Actor, ActorSystem, Props}

import scala.util.Random
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global

case object Produce

object Application extends App {
  val actorSystem = ActorSystem()
  val actor = actorSystem.actorOf(Props[Parent])

  Console.println("Press enter")
  Console.readLine() //I need a few seconds to attach VisualVM to the process ;)

  actorSystem.scheduler.schedule(2 seconds, 200 millis, actor, Produce)
}

class Parent extends Actor {
  override def receive: Receive = {
    case Produce =>
      val child = context.system.actorOf(Props[Child])
      child ! Produce
  }
}

class Child extends Actor {
  val data = new Array[Byte](20000000)

  override def receive: Receive = {
    case Produce =>
      Random.nextBytes(data)
  }
}

When I looked at the memory usage chart in VisualVM I saw that:

Application crashing because of OutOfMemoryError
Application crashing because of OutOfMemoryError

And obviously the error I expected:

1
2
3
4
5
6
7
8
9
10
java.lang.OutOfMemoryError: Java heap space
 at Child.<init>(Application.scala:28)
 at sun.reflect.GeneratedConstructorAccessor2.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at java.lang.Class.newInstance(Class.java:442)
 at akka.util.Reflect$.instantiate(Reflect.scala:44)
 at akka.actor.NoArgsReflectConstructor.produce(IndirectActorProducer.scala:105)
 at akka.actor.Props.newActor(Props.scala:213)
...

Great, I managed to reproduce the problem in a very, very simple way ;)

I limited the application memory to only 1GB. Note that the chart does not show 1GB of used memory. The application crashes when it tries to allocate memory, so the chart shows the last value before the crash.

Hypothesis

If the actor system removes references to an actor when the actor is stopped, garbage collector will remove the object from memory and the application will not crash. I should notice GC activity on VisualVM charts.

All I had to do, was sending a PoisonPill to the Child actor after generating the byte array:

1
2
3
4
5
6
7
8
9
class Child extends Actor {
  val data = new Array[Byte](20000000)

  override def receive: Receive = {
    case Produce =>
      Random.nextBytes(data)
      context.system.scheduler.scheduleOnce(1 second, self, PoisonPill)
  }
}

Now the program not only keeps running without any errors but also its memory usage chart looks as expected.

Garbage collector removes stopped actors from memory
Garbage collector removes stopped actors from memory

Conclusion

  • Do not forget about references stored internally by the actor system.

  • When you want to get rid of an Akka actor, simply stop it (if you do not keep any references to the actor instance).

  • Some of the actors you create may not be reusable. They won’t magically disappear, you have to remove them.


Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you watch programming live streams, check out my YouTube channel.
You can also follow me on Twitter: @mikulskibartosz

If you want to hire me, send me a message on LinkedIn or Twitter.


Bartosz Mikulski
Bartosz Mikulski * data scientist / software/data engineer * conference speaker * organizer of School of A.I. meetups in Poznań * co-founder of Software Craftsmanship Poznan & Poznan Scala User Group