As I discussed already earlier, we are (semi-happily) using Apache Zeppelin as Spark notebook. However, at some point Zeppelin notebooks were so slowly responding and running into time-out errors, that it was impossible to work with. Restarting the Zeppelin server did not help -- and for quite some time we were clueless what suddenly happened. At some point we figures out that Zeppelin has severe problems shutting down processes when errors occurred -- and starts accumulating zombie processes over time. We had a couple of hundred, that cluttered our system. Killing these zombie processes and restarting Zeppelin server did the trick -- now everything is running as smooth as before.
Thursday, October 6, 2016
Zeppelin Zombies ...
As I discussed already earlier, we are (semi-happily) using Apache Zeppelin as Spark notebook. However, at some point Zeppelin notebooks were so slowly responding and running into time-out errors, that it was impossible to work with. Restarting the Zeppelin server did not help -- and for quite some time we were clueless what suddenly happened. At some point we figures out that Zeppelin has severe problems shutting down processes when errors occurred -- and starts accumulating zombie processes over time. We had a couple of hundred, that cluttered our system. Killing these zombie processes and restarting Zeppelin server did the trick -- now everything is running as smooth as before.
Labels:
spark,
technology
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment