https://bugs.lttng.org/https://bugs.lttng.org/themes/lttng/favicon/a.ico?14249722912016-06-03T23:41:53ZLTTng bugs repositoryLTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30042016-06-03T23:41:53ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Feedback</i></li></ul><p>Hi,</p>
<p>Were you able to reproduce/observe on stable v2.8?</p>
<p>Cheers</p> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30052016-06-04T07:38:33ZFlorea Irinelfmarianirinel@gmail.com
<ul></ul><p>Hi,</p>
<p>Yes, I did just now on the ubuntu machine. Same behaviour.</p>
<p>Regards.</p> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30122016-06-13T12:29:44ZFlorea Irinelfmarianirinel@gmail.com
<ul></ul><p>Hi,</p>
<p>I've been debugging a bit. Within the sessiond_poll thread ,on receiving command 11 - LTTNG_CONSUMER_CLOSE_METADATA, the wait fd/poll pipe of the metadata stream is not closed when using snapshot sessions. Because the channel key passed to close_metadata() ( calling consumer_find_channel() and passing the channel ref to lttng_ustconsumer_close_metadata() ) corresponds to a channel for which the condition !channel->metadata_stream ( within lttng_ustconsumer_close_metadata() ) evaluates as true. Causing the metadata_poll thread to not get any LPOLLHUP event over the pipe and it doesn't call consumer_del_metadata_stream() to free the shm_table.<br />Does any of this make any sense? Do you think this is possibly the reason for the behaviour I'm getting?</p>
<p>Regards.</p> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30132016-06-13T14:19:46ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<ul></ul><p>Florea Irinel wrote:</p>
<blockquote>
<p>Hi,</p>
<p>I've been debugging a bit.</p>
</blockquote>
<p>Good!</p>
<blockquote>
<p>Within the sessiond_poll thread ,on receiving command 11 - LTTNG_CONSUMER_CLOSE_METADATA, the wait fd/poll pipe of the metadata stream is not closed when using snapshot sessions. Because the channel key passed to close_metadata() ( <br />calling consumer_find_channel() and passing the channel ref to lttng_ustconsumer_close_metadata() ) corresponds to a channel for which the condition !channel->metadata_stream ( within lttng_ustconsumer_close_metadata() ) evaluates as<br />true. Causing the metadata_poll thread to not get any LPOLLHUP event over the pipe and it doesn't call consumer_del_metadata_stream() to free the shm_table.<br />Does any of this make any sense? Do you think this is possibly the reason for the behaviour I'm getting?</p>
</blockquote>
<p>Well, were you able to fix your issue ? If so do you have a patch we can look at?</p>
<p>Since you have a reproducer I would suggest that you poke around and try a fix while we find some time to look at it.</p>
<p>Cheers</p>
<blockquote>
<p>Regards.</p>
</blockquote> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30202016-06-16T11:20:04ZFlorea Irinelfmarianirinel@gmail.com
<ul></ul><p>Hi,</p>
<p>Since on snapshot sessions the metadata stream is not monitored by the metadata thread the stream has to be freed somewhere else (the closing of the wait fd's does not suffice). Do you agree? If yes, what is the best approach :<br /> - to make sessiond to issue a LTTNG_CONSUMER_DESTROY_CHANNEL command after the LTTNG_CONSUMER_CLOSE_METADATA?<br /> - in consumerd at receiving LTTNG_CONSUMER_CLOSE_METADATA if the metadata stream is not monitored , notify channel thread using notify_thread_del_channel() ?<br /> - something else.</p>
<p>Regards.</p> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30212016-06-16T11:21:28ZFlorea Irinelfmarianirinel@gmail.com
<ul></ul><p>Florea Irinel wrote:</p>
<blockquote>
<p>Hi,</p>
<p>Since on snapshot sessions the metadata stream is not monitored by the metadata thread the stream has to be freed somewhere else (the closing of the wait fd's does not suffice). Do you agree? If yes, what is the best approach :<br />- to make sessiond to issue a LTTNG_CONSUMER_DESTROY_CHANNEL command after the LTTNG_CONSUMER_CLOSE_METADATA if the session is in snapshot mode?<br />- in consumerd at receiving LTTNG_CONSUMER_CLOSE_METADATA if the metadata stream is not monitored , notify channel thread using notify_thread_del_channel() ?<br />- something else.</p>
<p>Regards.</p>
</blockquote> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30252016-06-21T13:31:06ZFlorea Irinelfmarianirinel@gmail.com
<ul><li><strong>File</strong> <a href="/attachments/387">0001-Fix-consumerd-memory-leak-when-using-snapshot-sessio.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/387/0001-Fix-consumerd-memory-leak-when-using-snapshot-sessio.patch">0001-Fix-consumerd-memory-leak-when-using-snapshot-sessio.patch</a> added</li></ul><p>Hi,</p>
<p>Went with the second approach and created the patch attached. The memory allocated when starting tracing on snapshot sessions is now freed when destroying the session. From what I tested it does not interfere with other modes of sessions. Please let me know what you make of this .</p> LTTng-tools - Bug #1017: consumerd memory leakhttps://bugs.lttng.org/issues/1017?journal_id=30262016-06-21T14:08:46ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<ul></ul><p>Hi Florea,</p>
<p>I would recommend that you send a proper email to the mailing with the patch inline with a more verbose message explaining everything in the commit message (valgrind report, causes etc.) Please read <a class="external" href="http://lttng.org/community/#contributors-guide">http://lttng.org/community/#contributors-guide</a> for more detail.</p>
<p>I'm very glad that you went the patch way! We are always looking for new contributors.</p>
<p>Cheers!</p>