As I discussed already earlier, we are (semi-happily) using Apache Zeppelin as Spark notebook. However, at some point Zeppelin notebooks were so slowly responding and running into time-out errors, that it was impossible to work with. Restarting the Zeppelin server did not help -- and for quite some time we were clueless what suddenly happened. At some point we figures out that Zeppelin has severe problems shutting down processes when errors occurred -- and starts accumulating zombie processes over time. We had a couple of hundred, that cluttered our system. Killing these zombie processes and restarting Zeppelin server did the trick -- now everything is running as smooth as before.
No comments:
Post a Comment