https://bugs.lttng.org/https://bugs.lttng.org/themes/lttng/favicon/a.ico?14249722912021-06-17T17:00:30ZLTTng bugs repositoryBabeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39172021-06-17T17:00:30ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/3917/diff?detail_id=3684">diff</a>)</li></ul><p>Hi,</p>
<p>Is there any error messages on lttng-relayd or lttng-sessiond side?</p>
<p>Could you provide us with a bit more context on how the tracing session is configured?</p>
<p>Is "lttng clear" used?</p>
<p>Cheers</p> Babeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39182021-06-24T23:13:21ZSergei Dyshel
<ul></ul><p>No other error messages.</p>
<p>babeltrace2 crashed several times until our monitoring script detected it and restarted the whole chain of lttng-sessiond/relayd/babeltrace2.</p>
<p>We're using LTTNG framework for continuous low-latency tracing, i.e. our service constantly emits traces which are then processed live <br />by our babeltrace2 plugin.</p>
<p>The session is created with these commands:<br /><pre>
lttng create XXX --live
lttng enable-channel channel0 -u -s XXX --subbuf-size 1M --num-subbuf 8 --overwrite --tracefile-size 104857600 --tracefile-count 3
lttng enable-event 'XXX_*' -u -s XXX -c channel0
lttng add-context -u -t vpid -t vtid -s XXX -c channel0
lttng start XXX
</pre></p>
<p>relayd runs with this command:<br /><pre>
lttng-relayd --output /var/log/lttng-XXX/live
</pre></p>
<p>babeltrace2 runs with:<br /><pre>
babeltrace2 lttng-live net://localhost/host/DIRECTORY/XXX --component XXX:sink.XXX.view --log-level WARN
</pre></p> Babeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39192021-06-25T16:42:34ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<ul></ul><p>Do you see any pattern for when the error occurs? (i.e tracing is ok for over 2 hours etc, only happens on hooking babeltrace2 and happens 1 time in a 100 etc.)</p>
<p>That could help us pinpoint the problem.</p> Babeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39202021-07-14T00:33:28ZSergei Dyshel
<ul></ul><p>No pattern, the error happened only once.</p> Babeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39322021-08-16T19:14:11ZJérémie Galarneaujeremie.galarneau@efficios.com
<ul></ul><p>In the interest of reproducing your overall setup, can you give us some rough indication of the throughput of the application(s) being traced?</p>
<p>Are the network links between the consumer and relay daemons, or between the relay daemon and viewer severely congested/throttled?</p>
<p>Can you reproduce these crashes with the (default) text sink?</p> Babeltrace - Bug #1319: Assert in emit_inactivity_messagehttps://bugs.lttng.org/issues/1319?journal_id=39332021-08-30T20:29:35ZSergei Dyshel
<ul></ul><p>I couldn't reproduce the crash in many runs. Seems one-off issue.</p>
<p>What kind of throughput would you want me to measure? The application produces (usually) no more than 25MB of trace messages per hour.</p>