LTTng bugs repository: Issueshttps://bugs.lttng.org/https://bugs.lttng.org/themes/lttng/favicon/a.ico?14249722912022-03-16T20:23:07ZLTTng bugs repository
Redmine LTTng-tools - Feature #1349 (Resolved): Add '--all' to lttng starthttps://bugs.lttng.org/issues/13492022-03-16T20:23:07ZMichael Jeansonmjeanson@efficios.com
<p>Add a '--all' option to the 'lttng start' command that would start all configured sessions.</p> LTTng-UST - Bug #1310 (Resolved): Update man pages for API 1https://bugs.lttng.org/issues/13102021-04-21T14:18:32ZMichael Jeansonmjeanson@efficios.comLTTng-UST - Bug #1309 (Resolved): Add tests for compat API 0https://bugs.lttng.org/issues/13092021-04-21T14:16:10ZMichael Jeansonmjeanson@efficios.comLTTng-UST - Bug #1257 (Resolved): lttng-ust-java-tests failure on latest masterhttps://bugs.lttng.org/issues/12572020-04-20T19:39:44ZMichael Jeansonmjeanson@efficios.com
<p>Since the introduction of 'Refactor representation of nested types' in the master branch of ust and tools, the java agent test suite fails at 'org.lttng.ust.agent.integration.context.JulAppContextIT#testContextValueFloat'.</p>
<p>The output of the test suite shows the JVM dying on a double free ( <a href="https://ci.lttng.org/view/LTTng-ust/job/lttng-ust-java-tests_master_build/2119/arch=amd64,java_version=java-11-openjdk,liburcu_version=master,lttng_tools_version=master,lttng_ust_version=master/console" class="external">Full log</a> ) :<br /><pre>
06:46:43 Running org.lttng.ust.agent.integration.context.JulAppContextIT#testContextValueFloat
06:46:43 $ lttng create 8096ae8b-c7bb-4255-9664-924a62fb4948
06:46:43 Session 8096ae8b-c7bb-4255-9664-924a62fb4948 created.
06:46:43 Traces will be output to /home/jenkins/lttng-traces/8096ae8b-c7bb-4255-9664-924a62fb4948-20200419-104643
06:46:43 (returned from command)
06:46:43 AbstractLttngAgent: Initializing Agent for domain: JUL
06:46:43 LttngTcpSessiondClient: (root=true, domain=3) Connecting to sessiond
06:46:43 LttngTcpSessiondClient: (root=false, domain=3) Connecting to sessiond
06:46:43 LttngTcpSessiondClient: (root=false, domain=3) Registering to sessiond
06:46:43 LttngTcpSessiondClient: (root=false, domain=3) Waiting on sessiond commands...
06:46:43 LttngTcpSessiondClient: (root=false, domain=3) Registration done
06:46:46 LttngTcpSessiondClient: (root=true, domain=3) Connecting to sessiond
06:46:46 $ lttng enable-event -j -a -s 8096ae8b-c7bb-4255-9664-924a62fb4948
06:46:46 LttngTcpSessiondClient: (root=false, domain=3) Received enable event command: SessiondEnableEventCommand[eventName=*, logLevel=LTTNG_EVENT_LOGLEVEL_ALL, filterString=]
06:46:46 LttngTcpSessiondClient: (root=false, domain=3) Sending response: LttngAgentResponse[code=1, sucess]
06:46:46 All JUL events are enabled in channel channel0
06:46:46 (returned from command)
06:46:46 $ lttng add-context -j -t $app.Retriever1:some.context_name -s 8096ae8b-c7bb-4255-9664-924a62fb4948
06:46:46 LttngTcpSessiondClient: (root=false, domain=3) Received enable app-context command
06:46:46 LttngTcpSessiondClient: (root=false, domain=3) Sending response: LttngAgentResponse[code=1, sucess]
06:46:46 JUL context $app.Retriever1:some.context_name added to all channels
06:46:46 (returned from command)
06:46:46 $ lttng start 8096ae8b-c7bb-4255-9664-924a62fb4948
06:46:46 libust[1265686/1265707]: Message Received "Create Session" (64), Handle "root" (0) (in print_cmd() at lttng-ust-comm.c:467)
06:46:46 libust[1265686/1265707]: Return value: 1 (in handle_message() at lttng-ust-comm.c:1085)
06:46:46 libust[1265686/1265707]: message successfully sent (in send_reply() at lttng-ust-comm.c:643)
06:46:46 libust[1265686/1265707]: Message Received "Create Channel" (81), Handle "session" (1) (in print_cmd() at lttng-ust-comm.c:467)
...
06:46:46 libust[1265686/1265707]: Message Received "Enable" (128), Handle "session" (1) (in print_cmd() at lttng-ust-comm.c:467)
06:46:46 libust[1265686/1265707]: Sent register event notification for name "lttng_jul:event": ret_code 0, event_id 0
06:46:46 (in ustcomm_register_event() at lttng-ust-comm.c:1517)
06:46:46 liblttng_ust_tracepoint[1265686/1265707]: Registering probe to tracepoint lttng_jul:event. Queuing release. (in __tracepoint_probe_register_queue_release() at tracepoint.c:652)
06:46:46 liblttng_ust_tracepoint[1265686/1265707]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at tracepoint.c:743)
06:46:46 libust[1265686/1265707]: Sent register enum notification for name "dynamic_type_enum": ret_code 0
06:46:46 (in ustcomm_register_enum() at lttng-ust-comm.c:1622)
06:46:46 double free or corruption (!prev)
06:46:46 Aborted
</pre></p>
<p>The sessiond log reports an error serialising the metadata ( <a href="https://ci.lttng.org/view/LTTng-ust/job/lttng-ust-java-tests_master_build/2119/arch=amd64,java_version=java-11-openjdk,liburcu_version=master,lttng_tools_version=master,lttng_ust_version=master/artifact/lttng-sessiond.log" class="external">Full log</a> ) :<br /><pre>
DEBUG3 - 10:46:46.312634981 [1265363/1265375]: Append to metadata: "floating_point { exp_dig = 8; mant_dig = 24; align = 8; } _float;
" (in lttng_metadata_printf() at ust-metadata.c:166)
Error: Error appending channel metadata (errno = -75)
...
Error: 1 trace chunks are leaked by lttng-consumerd. This can be caused by an internal error of the session daemon.
Error: consumer closed the command socket
Error: Health error occurred in thread_consumer_management
</pre></p>
<p>Running gdb on the jvm during the test shows the double free happening in 'ustcomm_register_channel' on 'fields'<br /><pre>
0 0x00007f1dd6b68ce5 in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1 0x00007f1dd6b52857 in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2 0x00007f1dd6bac2b0 in __libc_message () from /usr/lib/libc.so.6
No symbol table info available.
#3 0x00007f1dd6bb374a in malloc_printerr () from /usr/lib/libc.so.6
No symbol table info available.
#4 0x00007f1dd6bb55cc in _int_free () from /usr/lib/libc.so.6
No symbol table info available.
#5 0x00007f1da04d94e7 in ustcomm_register_channel (sock=15, session=0x7f1d5c000c70, session_objd=1, channel_objd=2, nr_ctx_fields=1, ctx_fields=0x7f1d5c013f80, chan_id=0x7f1d997fcdb8, header_type=0x7f1d5c013db8) at lttng-ust-comm.c:1697
len = 11960
msg = {header = {notify_cmd = 1}, m = {session_objd = 1, channel_objd = 2, ctx_fields_len = 11960, padding = '\000' <repeats 31 times>}}
reply = {header = {notify_cmd = 1818388844}, r = {ret_code = 1735292020, chan_id = 1953723743, header_type = 1634890847, padding = "cepoint[185588/185609]: Release "}}
fields_len = 11960
fields = 0x7f1d5c01c2e0
ret = 0
nr_write_fields = 13
__func__ = "ustcomm_register_channel"
#6 0x00007f1da04ef03c in lttng_session_enable (session=0x7f1d5c000c70) at lttng-events.c:458
ctx = 0x7f1d5c013f50
fields = 0x7f1d5c013f80
nr_fields = 1
chan_id = 4294967295
ret = 0
chan = 0x7f1d5c013d70
notify_socket = 15
__func__ = "lttng_session_enable"
#7 0x00007f1da04e46d7 in lttng_session_cmd (objd=1, cmd=128, arg=139765106071992, uargs=0x7f1d997fd290, owner=0x7f1da0575c00 <local_apps>) at lttng-ust-abi.c:580
session = 0x7f1d5c000c70
</pre></p>
<p>To reproduce the problem, build the latest master from ust and tools with the java agents enabled and debug symbols. Checkout the <a href="https://github.com/lttng/lttng-ust-java-tests" class="external">lttng-ust-java-test</a> repo and use the following script to run the test, the only additionnal requirement on top of the build dependencies is 'maven' which you should be packaged in most distro :</p>
<pre><code class="shell syntaxhl" data-language="shell"><span class="c">#!/bin/sh</span>
<span class="nv">PREFIX</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/opt"</span>
<span class="nv">LOG4J_JAR</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span><span class="s2">/share/java/log4j-1.2.17.jar"</span>
<span class="c"># To override the default java jvm</span>
<span class="c">#export JAVA_HOME=/usr/lib/jvm/${java_version}-${arch}</span>
<span class="nb">export </span><span class="nv">LD_LIBRARY_PATH</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/lib
<span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/bin:<span class="k">${</span><span class="nv">PATH</span><span class="k">}</span>
<span class="nb">export </span><span class="nv">LTTNG_UST_DEBUG</span><span class="o">=</span>1
<span class="nb">export </span><span class="nv">LTTNG_CONSUMERD32_BIN</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/lib/lttng/libexec/lttng-consumerd
<span class="nb">export </span><span class="nv">LTTNG_CONSUMERD64_BIN</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/lib/lttng/libexec/lttng-consumerd
<span class="nb">export </span><span class="nv">LTTNG_SESSION_CONFIG_XSD_PATH</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/share/xml/lttng
<span class="c"># Start the lttng-sessiond</span>
lttng-sessiond <span class="nt">-b</span> <span class="nt">-vvv</span> <span class="o">></span>lttng-sessiond.log 2>&1
mvn <span class="se">\</span>
<span class="nt">-Dmaven</span>.test.failure.ignore<span class="o">=</span><span class="nb">true</span> <span class="se">\</span>
<span class="nt">-DfailIfNoTests</span><span class="o">=</span><span class="nb">false</span> <span class="se">\</span>
<span class="nt">-Dlog4j-jar-location</span><span class="o">=</span><span class="k">${</span><span class="nv">LOG4J_JAR</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-Dcommon-jar-location</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/share/java/lttng-ust-agent-common.jar <span class="se">\</span>
<span class="nt">-Djul-jar-location</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/share/java/lttng-ust-agent-jul.jar <span class="se">\</span>
<span class="nt">-Dlog4j-jar-location</span><span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/share/java/lttng-ust-agent-log4j.jar <span class="se">\</span>
<span class="nt">-DargLine</span><span class="o">=</span><span class="nt">-Djava</span>.library.path<span class="o">=</span><span class="k">${</span><span class="nv">PREFIX</span><span class="k">}</span>/lib <span class="se">\</span>
<span class="nt">-Dtest</span><span class="o">=</span><span class="s1">'JulAppContextOrderingIT#testSessionAgentRetriever'</span> <span class="se">\</span>
clean verify
<span class="c"># Kill the sessiond</span>
killall lttng-sessiond
</code></pre>
<p>The only tricky part is attaching gdb to the correct jvm, I haven't yet found a reliable way to fully automate this step. You need to run the previouly mentionned script and wait for the final phase of the test to start before starting gdb. You have to wait for the second time you see coloured output before attaching gdb.</p>
<p>Attach gdb to the jvm running the test (disable signals used internally by the jvm)<br /><pre>
gdb --init-eval-command='handle SIGSEGV nostop noprint pass' --init-eval-command='handle SIG62 nostop noprint pass' -p $(jps | grep -i sure | awk '{ print $1 }')
</pre></p> Babeltrace - Bug #1120 (Resolved): g_ptr_array_insert() requires glib2 >= 2.40https://bugs.lttng.org/issues/11202017-06-13T20:39:51ZMichael Jeansonmjeanson@efficios.com
<p>In 'plugins/ctf/fs-src/fs.c' we use 'g_ptr_array_insert()' which requires a glib2 version of at least 2.40, as documented here :</p>
<pre><code><a class="external" href="https://developer.gnome.org/glib/stable/glib-Pointer-Arrays.html#g-ptr-array-insert">https://developer.gnome.org/glib/stable/glib-Pointer-Arrays.html#g-ptr-array-insert</a></code></pre>
<p>Our current target is glib2 >= 2.22, should we bump this requirement or replace g_ptr_array_insert?</p> LTTng - Bug #1056 (Resolved): Codename for LTTng 2.9https://bugs.lttng.org/issues/10562016-08-03T21:50:58ZMichael Jeansonmjeanson@efficios.com
<p>What should be the codename of the LTTng 2.9 release?</p>
<p>Must be a micro-brewed Quebec beer and start with a "J".</p>