LTTng bugs repository: Issueshttps://bugs.lttng.org/https://bugs.lttng.org/themes/lttng/favicon/a.ico?14249722912024-03-13T17:07:08ZLTTng bugs repository
Redmine LTTng-tools - Bug #1411 (Feedback): Memory leak when relay daemon exits before application startshttps://bugs.lttng.org/issues/14112024-03-13T17:07:08ZMikael Beckius
<p>When the relay daemon is shutdown after creating a live session but before applications are started the shared memory allocated for tracing appears to remain and new memory is allocated for every application start.</p>
<p>How to reproduce:<br />host:~# lttng create micke --live<br />Spawning a session daemon<br />Spawning a relayd daemon<br />Live session micke created.<br />Traces will be output to tcp4://127.0.0.1:5342/ [data: 5343]<br />Live timer interval set to 1000000 us</p>
<p>host:~# lttng enable-event --userspace --all<br />All ust events are enabled in channel channel0</p>
<p>host:~# lttng start<br />Tracing started for session micke</p>
<p>host:~# killall -9 lttng-relayd</p>
<p>host:~# free -h<br /> total used free shared buff/cache available<br />Mem: 15Gi 207Mi 15Gi 572Ki 320Mi 15Gi<br />Swap: 0B 0B 0B</p>
<p>host:~# ./micke-lttng<br />Mikael LTTNG 2015 - Starting<br />Mikael LTTNG 2015 - Signing out</p>
<p>host:~# free -h<br /> total used free shared buff/cache available<br />Mem: 15Gi 248Mi 15Gi 40Mi 360Mi 15Gi<br />Swap: 0B 0B 0B</p>
<p>host:~# ./micke-lttng<br />Mikael LTTNG 2015 - Starting<br />Mikael LTTNG 2015 - Signing out</p>
<p>host:~# free -h<br /> total used free shared buff/cache available<br />Mem: 15Gi 288Mi 15Gi 80Mi 400Mi 15Gi<br />Swap: 0B 0B 0B</p>
<p>host:~# lttng destroy micke<br />Destroying session micke..<br />Session micke destroyed</p>
<p>host:~# free -h<br /> total used free shared buff/cache available<br />Mem: 15Gi 289Mi 15Gi 80Mi 400Mi 15Gi<br />Swap: 0B 0B 0B<br />host:~#</p>
<p>Version:<br />lttng-tools 2.13.11</p>
<p>Analyzis:<br />It seems that when the first application of a session starts, after the relay daemon has been shutdown, a failure to transfer streams to the relay deamon triggers a clean up through a call to ust_consumer_destroy_channel. But it appears that the cleanup isn't complete and the channel reference count remains incremented. Decrementing the reference count appears to be blocked in clean_channel_stream_list by stream->monitor = 0; preventing CONSUMER_CHANNEL_DEL from reaching consumer_del_channel(chan);</p>
<p>Information has it that this problem is NOT reproduced on 2.13 but I haven't tested that myself</p> LTTng-tools - Bug #1392 (Feedback): lttng view: Stream 0 is not declared in metadatahttps://bugs.lttng.org/issues/13922023-09-27T20:38:17ZRicardo Nabinger Sanchezrnsanchez@gmail.com
<p>While trying to capture UST events from a simple program, even though kernel events seem to be properly captured, UST events are not. Trace Compass cannot decode the UST data (but it can decode from the kernel channel).<br />The application is single-threaded, performs only disk and stdout I/O, nothing fancy. Compiled with <code>-finstrument-functions</code>, no optimizations, and with debugging symbols.</p>
<p>Session setup:<br /><pre>
# lttng create
# lttng enable-channel -u --subbuf-size=64M --num-subbuf=8 chan_ust
# lttng enable-channel -k --subbuf-size=64M --num-subbuf=2 chan_kernel
# lttng add-context -u -t vpid -t ip -t procname -t vtid -c chan_ust
# lttng enable-event -u -a -c chan_ust
# lttng enable-event -k -c chan_kernel --syscall --all
# lttng enable-event -k -c chan_kernel lttng_statedump_block_device,lttng_statedump_file_descriptor,lttng_statedump_process_state,mm_page_alloc,mm_page_free,net_dev_xmit,netif_receive_skb,sched_pi_setprio,sched_process_exec,sched_process_fork,sched_switch,sched_wakeup,sched_waking,softirq_entry,softirq_exit,softirq_raise
# lttng start
(possibly excessive preloads, but these are from my notes, collecting data on Varnish Cache)
# LTTNG_UST_BLOCKING_RETRY_TIMEOUT=-1 LD_PRELOAD=liblttng-ust-fd.so:liblttng-ust-fork.so:liblttng-ust-dl.so:liblttng-ust-cyg-profile.so:liblttng-ust-libc-wrapper.so:liblttng-ust-pthread-wrapper.so tests/test-playlist tests/playlist/bug_ffmpeg_1000-x20.m3u
# lttng stop
# lttng view > /dev/null
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
[error] Stream 0 is not declared in metadata.
</pre></p>
Environment:
<ul>
<li>lttng-modules-2.13.10 (from tarball)</li>
<li>lttng-tools-2.13.11 (from tarball)</li>
<li>lttng-ust-2.13.6 (from tarball)</li>
<li>babeltrace (git; at <code>v1.5.11</code>)</li>
<li>Linux kernel 6.6.0-rc1</li>
<li>Slackware64-current, up-to-date as of 2023-09-26</li>
<li>Trace Compass 9.1.0.202309200833</li>
</ul>
<p>IRC snippet:<br /><pre>
Sep/27 16:13:54 rnsanchez I'm trying to find out what I'm doing wrong.. I keep getting "[error] Stream 0 is not declared in metadata." in my captures. I guess this is making tracecompass not being able to show UST events (simple function enter-exit instrumentation)
Sep/27 16:16:46 rnsanchez using my old notes, while checking https://lttng.org/docs/v2.13/#doc-liblttng-ust-cyg-profile and other sections
Sep/27 16:18:25 jgalar rnsanchez: shot in the dark, but are you making sure the session is stopped or destroyed before reading the trace?
Sep/27 16:18:28 rnsanchez if it helps knowing, events from kernel channel are fine and I can see them in tracecompass (the trace is huge)
Sep/27 16:18:52 rnsanchez jgalar: yes, with lttng stop, then I copy the data from root to my user
Sep/27 16:19:10 rnsanchez but I get that Stream 0 regardless.. it's from lttng view
Sep/27 16:20:11 jgalar it sounds like a metadata flushing issue, any chance you can share the trace? (or at least, the metadata files)
Sep/27 16:20:42 rnsanchez yes, sure. just a moment while I upload it
Sep/27 16:20:48 jgalar okay thanks
Sep/27 16:21:43 ebugden not sure if related, but it's triggering deja vu of this issue: https://github.com/efficios/barectf/commit/d024537859c1d869bfa1cedc8abe8e3f7a648faa
Sep/27 16:22:23 ebugden maybe a similar problem in a similar area in lttng view
Sep/27 16:25:28 rnsanchez jgalar: https://rnsanchez.wait4.org/auto-20230927-161138.tar.xz
Sep/27 16:27:33 rnsanchez jgalar: this last one has incomplete events as I was trying to investigate, but I can bundle another one with full events as I used to collect
Sep/27 16:31:51 rnsanchez ebugden: is there anything on my side I can do?
Sep/27 16:34:38 ebugden rnsanchez: i don't feel i'm familiar enough with this situation to say; i was tossing out the association as possible food for thought for jgalar
Sep/27 16:35:12 rnsanchez oh ok :-)
Sep/27 16:43:03 rnsanchez jgalar: just in case, here is with full events: https://rnsanchez.wait4.org/auto-20230927-164014.tar.xz
Sep/27 16:45:53 jgalar ebugden: yup, the errors are indeed similar
Sep/27 16:50:59 jgalar hmm, the user space trace's metadata doesn't declare any stream class, that's... unexpected...
Sep/27 16:51:14 jgalar which versions of lttng-ust and lttng-tools are you running?
Sep/27 16:55:03 rnsanchez ust 2.13.6 (tarball) and tools 2.13.11 (tarball too)
Sep/27 16:55:07 rnsanchez compiled today
Sep/27 16:55:13 jgalar okay
Sep/27 17:17:57 ebugden rnsanchez (cc: jgalar): would you open a bug report on https://bugs.lttng.org/ ?
Sep/27 17:18:16 rnsanchez ebugden: sure
Sep/27 17:19:21 ebugden thanks! (we have a few hypotheses, but we'd need more information about exactly how the trace is generated)
Sep/27 17:19:58 jgalar rnsanchez: it's weird, the user space trace's stream files just have the packet headers... if you can reproduce the problem while running lttng-sessiond with the `-vvv --verbose-consumer` options to get the logs, it would be helpful
Sep/27 17:20:47 rnsanchez I can try. I suppose I have to finish the sessiond already running and manually launch another?
Sep/27 17:21:06 jgalar yep
</pre></p> LTTng - Bug #1385 (Feedback): [lttng-live] error: recevie a message in the pasthttps://bugs.lttng.org/issues/13852023-08-16T18:53:52ZBin Yuan
<p>My program works well most of time. But in rare case, it reports the following hightling error.<br /><img src="https://bugs.lttng.org/attachments/download/572/clipboard-202308170250-vfley.jpg" alt="" loading="lazy" /></p>
<p>What will lead to those branch? any suggestion to debug?</p> LTTng - Bug #1370 (Confirmed): Why "lttng create --live" spawns a local relay daemon but not in ...https://bugs.lttng.org/issues/13702023-04-06T15:33:05ZBin Yuan
<p>The relayd spawned by lttng-create command don't close the file descriptor such like stdout.<br />Why not spawn the relayd with "--daemonize" option.</p> LTTng - Bug #1369 (Feedback): Nano clock value overflows the signed 64-bit integer range on the b...https://bugs.lttng.org/issues/13692023-03-26T02:41:32ZBin Yuan
<p>I launched a program producing traces to the lttng consumerd and lttng relayd, also launching a babeltrace command as lttng live consumer to show the tracing real-timely.</p>
<p>It works well when the channel is set to per user mode. But when it is changed to per process per user without no other changes, the babeltrace2 client crashing showing a overflow error:</p>
<p>The last error message of babeltrace2 exception stack shows :</p>
<p>"Cannot convert cycle to nanoseconds from origin for given clock class: value overflows the signed 64-bit integer range: cc-addr=****, cc-name="monotonic", cc-freq=1000000000, ....."</p>
<p>What's the difference resulting this crash ? How to solve it since the perprocess per user mode is perfered for me.</p>
<p>version info:<br />babeltrace2: 2.0.4<br />lttng comands: 2.12.12</p> LTTng-tools - Bug #1362 (New): Listing a trigger with an event rule matches condition with a upro...https://bugs.lttng.org/issues/13622022-10-24T21:10:59ZJérémie Galarneaujeremie.galarneau@efficios.com
<p>Running against <code>d67dc273f</code>.</p>
<p>When listing triggers, I get a crash:</p>
<pre>
[root@carbonara lttng-tools]# /tmp/lttng/bin/lttng list-triggers
- name: uprobe_trigger
owner uid: 0
condition: event rule matches
rule: uprobe_trigger (type: kernel:uprobe, location type: ELF, location: tests/regression/tools/notification//../../..//utils/testapp/userspace-probe-elf-binary/.libs/userspace-probe-elf-binary:test_function)
errors: none
action:notify
lttng: ../../src/common/dynamic-array.hpp:53: void* lttng_dynamic_array_get_element(const lttng_dynamic_array*, size_t): Assertion `element_index < array->size' failed.
Aborted (core dumped)
</pre>
<p>gdb output:<br /><pre>
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `/tmp/lttng/bin/lttng list-triggers'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007ffbb3aa164c in ?? () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007ffbb3aa164c in ?? () from /usr/lib/libc.so.6
#1 0x00007ffbb3a51958 in raise () from /usr/lib/libc.so.6
#2 0x00007ffbb3a3b53d in abort () from /usr/lib/libc.so.6
#3 0x00007ffbb3a3b45c in ?? () from /usr/lib/libc.so.6
#4 0x00007ffbb3a4a486 in __assert_fail () from /usr/lib/libc.so.6
#5 0x0000564a49b9d130 in lttng_dynamic_array_get_element (array=0x564a4ab6c040, element_index=0) at ../../src/common/dynamic-array.hpp:53
#6 0x0000564a49b9d378 in lttng_action_path_copy (src=0x564a4ab6c040, dst=0x564a4ab6c540) at actions/path.cpp:110
#7 0x0000564a49b4d505 in lttng_error_query_action_create (trigger=0x564a4ab6b5d0, action_path=0x564a4ab6c040) at error-query.cpp:232
#8 0x0000564a49b3ee7c in print_action_errors (trigger=0x564a4ab6b5d0, action_path_indexes=0x0, action_path_length=0) at commands/list_triggers.cpp:765
#9 0x0000564a49b3f99a in print_one_action (trigger=0x564a4ab6b5d0, action=0x564a4ab6b4e0, action_path_indexes=0x0, action_path_length=0) at commands/list_triggers.cpp:965
#10 0x0000564a49b400c7 in print_one_trigger (trigger=0x564a4ab6b5d0) at commands/list_triggers.cpp:1123
#11 0x0000564a49b4036e in print_sorted_triggers (triggers=0x564a4ab6b110) at commands/list_triggers.cpp:1201
#12 0x0000564a49b40c92 in cmd_list_triggers (argc=0, argv=0x7fffa9404f18) at commands/list_triggers.cpp:1420
#13 0x0000564a49b438d6 in handle_command (argc=1, argv=0x7fffa9404f10) at lttng.cpp:238
#14 0x0000564a49b4402c in parse_args (argc=2, argv=0x7fffa9404f08) at lttng.cpp:427
#15 0x0000564a49b441a8 in main (argc=2, argv=0x7fffa9404f08) at lttng.cpp:476
</pre></p>
<p>This is a trigger created by the following test:<br /><code>tests/regression/tools/notification/test_notification_kernel_userspace_probe</code></p>
<p>It can be reproduced by stopping the test before invoking the event-generating application.</p> LTTng-modules - Bug #1358 (New): Failed to deploy lttng modules on NVIDIA jetson devicehttps://bugs.lttng.org/issues/13582022-09-13T13:22:48Zliuhonggang liu
<p>Hello, I installed lttng and lttng modules on NVIDIA Orin. <br />When using apt install, the results are as follows.<br /><pre><code class="shell syntaxhl" data-language="shell"><span class="c"># lttng list --kernel</span>
Error: Unable to list kernel events: Kernel tracer not available
</code></pre></p>
<pre><code class="shell syntaxhl" data-language="shell"><span class="c">#ps aux | grep lttng-sessiond</span>
root 52100 0.0 0.0 1022064 12736 ? Ssl 15:05 0:00 /usr/bin/lttng-sessiond
root 52101 0.0 0.0 41968 664 ? S 15:05 0:00 /usr/bin/lttng-sessiond
orin-d 62549 0.0 0.0 11640 684 pts/0 S+ 20:54 0:00 <span class="nb">grep</span> <span class="nt">--color</span><span class="o">=</span>auto lttng-sessiond
</code></pre>
<pre>
# dpkg -l | grep lttng
ii liblttng-ctl0:arm64 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility library
ii liblttng-ust-ctl4:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (trace control library)
ii liblttng-ust-dev:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (development files)
ii liblttng-ust-python-agent0:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (Python agent native library)
ii liblttng-ust0:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (tracing libraries)
ii lttng-modules-dkms 2.12.6-1~ubuntu20.04.1 all Linux Trace Toolkit (LTTng) kernel modules (DKMS)
ii lttng-tools 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility programs
ii python3-lttng 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility Python bindings
</pre>
<p>The device information is as follows.<br /><pre><code class="shell syntaxhl" data-language="shell"><span class="c"># uname -a</span>
Linux orind-d 5.10.65-tegra <span class="c">#2 SMP PREEMPT Thu Jun 16 18:24:26 CST 2022 aarch64 aarch64 aarch64 GNU/Linux</span>
</code></pre><br /><pre><code class="shell syntaxhl" data-language="shell"><span class="c"># cat /etc/os-release</span>
<span class="nv">NAME</span><span class="o">=</span><span class="s2">"Ubuntu"</span>
<span class="nv">VERSION</span><span class="o">=</span><span class="s2">"20.04.4 LTS (Focal Fossa)"</span>
<span class="nv">ID</span><span class="o">=</span>ubuntu
<span class="nv">ID_LIKE</span><span class="o">=</span>debian
<span class="nv">PRETTY_NAME</span><span class="o">=</span><span class="s2">"Ubuntu 20.04.4 LTS"</span>
<span class="nv">VERSION_ID</span><span class="o">=</span><span class="s2">"20.04"</span>
<span class="nv">HOME_URL</span><span class="o">=</span><span class="s2">"https://www.ubuntu.com/"</span>
<span class="nv">SUPPORT_URL</span><span class="o">=</span><span class="s2">"https://help.ubuntu.com/"</span>
<span class="nv">BUG_REPORT_URL</span><span class="o">=</span><span class="s2">"https://bugs.launchpad.net/ubuntu/"</span>
<span class="nv">PRIVACY_POLICY_URL</span><span class="o">=</span><span class="s2">"https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"</span>
<span class="nv">VERSION_CODENAME</span><span class="o">=</span>focal
<span class="nv">UBUNTU_CODENAME</span><span class="o">=</span>focal
</code></pre></p>
<p>At the same time, I tried the method of source code, such as the source code installation method introduced in <a class="external" href="https://lttng.org/docs/v2.13/">https://lttng.org/docs/v2.13/</a>.<br /><pre>
# dpkg -l | grep -e libuuid -e popt -e userspace -e libxml2
ii can-utils 2018.02.0-1ubuntu1 arm64 SocketCAN userspace utilities and tools
ii dmsetup 2:1.02.167-1ubuntu1 arm64 Linux Kernel Device Mapper userspace library
ii gvfs:arm64 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - GIO module
ii gvfs-backends 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - backends
ii gvfs-bin 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - deprecated command-line tools
ii gvfs-common 1.44.1-1ubuntu1 all userspace virtual filesystem - common data files
ii gvfs-daemons 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - servers
ii gvfs-fuse 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - fuse server
ii gvfs-libs:arm64 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - private libraries
ii libdevmapper1.02.1:arm64 2:1.02.167-1ubuntu1 arm64 Linux Kernel Device Mapper userspace library
ii libi2c0:arm64 4.1-2build2 arm64 userspace I2C programming library
ii libibverbs1:arm64 28.0-1ubuntu1 arm64 Library for direct userspace use of RDMA (InfiniBand/iWARP)
ii libnftnl11:arm64 1.1.5-1 arm64 Netfilter nftables userspace API library
ii libpopt-dev:arm64 1.16-14 arm64 lib for parsing cmdline parameters - development files
ii libpopt0:arm64 1.16-14 arm64 lib for parsing cmdline parameters
ii liburcu-dev:arm64 0.12.2-1~ubuntu20.04.2 arm64 userspace RCU (read-copy-update) library - development files
ii liburcu6:arm64 0.12.2-1~ubuntu20.04.2 arm64 userspace RCU (read-copy-update) library
ii libusb-1.0-0:arm64 2:1.0.23-2build1 arm64 userspace USB programming library
ii libusb-1.0-0-dev:arm64 2:1.0.23-2build1 arm64 userspace USB programming library development files
ii libuuid1:arm64 2.34-0.1ubuntu9.3 arm64 Universally Unique ID library
ii libxml2:arm64 2.9.10+dfsg-5ubuntu0.20.04.1 arm64 GNOME XML library
ii libxml2-dev:arm64 2.9.10+dfsg-5ubuntu0.20.04.1 arm64 Development files for the GNOME XML library
ii libxml2-utils 2.9.10+dfsg-5ubuntu0.20.04.3 arm64 XML utilities
ii network-manager 1.22.10-1ubuntu2.3 arm64 network management framework (daemon and userspace tools)
ii nvidia-l4t-optee 34.1.0-20220406120854 arm64 OP-TEE userspace daemons, test programs and libraries
ii python3-lxml:arm64 4.5.0-1ubuntu0.5 arm64 pythonic binding for the libxml2 and libxslt libraries
</pre></p>
<pre>
sudo ln -snf /usr/src/linux-headers-5.10.65-tegra-ubuntu20.04_aarch64/kernel-5.10 /lib/modules/5.10.65-tegra/build
</pre>
<pre>
# orin-d@orind-d:~/tmp/lttng-modules-2.13.5$ make
/home/orin-d/tmp/lttng-modules-2.13.5/src/wrapper/kallsyms.c:20:3: error: #error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
20 | # error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
| ^~~~~
make[2]: *** [scripts/Makefile.build:281: /home/orin-d/tmp/lttng-modules-2.13.5/src/wrapper/kallsyms.o] Error 1
make[1]: *** [Makefile:1852: /home/orin-d/tmp/lttng-modules-2.13.5/src] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.10.65-tegra-ubuntu20.04_aarch64/kernel-5.10'
make: *** [Makefile:31: modules] Error 2
</pre> LTTng-tools - Bug #1342 (New): Rotation thread's handling of session consumed size notifications ...https://bugs.lttng.org/issues/13422021-12-09T21:40:35ZJérémie Galarneaujeremie.galarneau@efficios.com
<p>Related to this change:<br /><a class="external" href="https://review.lttng.org/c/lttng-tools/+/6886/">https://review.lttng.org/c/lttng-tools/+/6886/</a></p>
The race described in that change has another consequence: the notification could refer to a different session instance of the same name:
<ul>
<li>notification emitted for session <code>foo</code>,</li>
<li>session <code>foo</code> is destroyed,</li>
<li>a new session named <code>foo</code> is created,</li>
<li>a rotation is triggered for <code>foo</code>.</li>
</ul>
There are multiple solutions for this:
<ul>
<li>Provide a unique session id as part the of the evaluation of <code>SESSION_CONSUMED_SIZE</code>,</li>
<li>Register an internal trigger that has the <code>rotate session</code> action (since ids are sampled during their handling).</li>
</ul>
<p>I can't work on a fix right now, but here is a placeholder patch that will eventually contain the fix:<br /><a class="external" href="https://review.lttng.org/c/lttng-tools/+/6907">https://review.lttng.org/c/lttng-tools/+/6907</a></p>
<p>This is hypothetical and I noticed it during a code review; nobody reported this problem.</p> LTTng - Bug #1315 (Feedback): Kernel panics after `pkill lttng`; root session daemon has active t...https://bugs.lttng.org/issues/13152021-05-07T20:09:26ZPhilippe Proulxeeppeliteloop@gmail.com
<p>I don't know how to reproduce this bug.</p>
<p>I played with the <code>lttng add-trigger</code> command while writing usage examples for its manual page.</p>
<p>I started the root session daemon as such:</p>
<pre>
# lttng-sessiond --daemonize --group=eepp
</pre>
<p>I then ran commands such as:</p>
<pre>
$ lttng add-trigger --name user --condition=event-rule-matches --domain=user --action=notify
</pre>
<pre>
$ lttng add-trigger --condition=event-rule-matches \
--domain=user --action=notify \
--rate-policy=every:10
</pre>
<pre>
$ lttng add-trigger --owner-uid=33 \
--condition=event-rule-matches \
--domain=kernel --name='sched*' \
--action=notify
</pre>
<pre>
$ lttng add-trigger --condition=event-rule-matches \
--domain=kernel --type=syscall \
--filter='fd < 3' \
--action=start-session my-session \
--rate-policy=once-after:40
</pre>
<p>Note that I had no tracing session named <code>my-session</code>.</p>
<p>After a few minutes, Xorg froze. I managed to login again in a virtual console. I ran:</p>
<pre>
# pkill lttng
</pre>
<p>and got an instant kernel panic.</p>
<p>Attached is a photo of what was on the screen after running <code>pkill</code>.</p>
<p>Using:</p>
<ul>
<li>LTTng-tools <code>60860e547ce31ea629e846e00b66342425474b8d</code>.</li>
<li>LTTng-UST <code>a0f2513af262a19822d46f84cd5e34be0badc484</code></li>
<li>LTTng-modules <code>51ef453614a6db2b778595b16d93283d25db974a</code></li>
<li>liburcu <code>5e1b7c840a2b21b8442b322cedbb70a790e49520</code></li>
</ul> LTTng - Feature #1269 (New): Document clearly the versioning scheme of lttng-projects.https://bugs.lttng.org/issues/12692020-05-22T19:42:08ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<p>Correct me if I am wrong but it is an "unwritten" rule that we follow semver [1]. Some organization might appreciate that we state clearly our versioning policy.</p>
<p>[1] <a class="external" href="https://semver.org/">https://semver.org/</a></p> LTTng - Feature #1268 (New): Adopting a Vulnerability Disclosure Policy for our projectshttps://bugs.lttng.org/issues/12682020-05-22T19:27:49ZJonathan Rajotte Julienjonathan.rajotte-julien@efficios.com
<p>Some organization might require/appreciate such policy.</p>
<p>We could probably base our policy on <a class="external" href="https://disclose.io/">https://disclose.io/</a>, considering they have "terms" [1] with canada in mind.</p>
<p>[1] <a class="external" href="https://github.com/disclose/disclose/tree/master/terms">https://github.com/disclose/disclose/tree/master/terms</a></p>
<p>IANAL</p> LTTng-tools - Feature #1180 (New): SDT tracing does not work when the probes are compiled with se...https://bugs.lttng.org/issues/11802019-03-27T15:41:21ZNaser Ez
<p>Tracing SDT probes does not work when the probes are compiled with semaphores.</p>
<p>For example, when we have a probe like:</p>
<pre>
Provider: nginx
Name: http__subrequest__start
Location: 0x0000000000429e9c, Base: 0x0000000000473810, Semaphore: 0x00000000006920ba
Arguments: 8@%rbx__
</pre>
<p>then running the following command:<br /><pre>
lttng enable-event -k nginx:http__subrequest__start --userspace-probe=sdt:/usr/local/nginx/sbin/nginx:nginx:http__subrequest__start
</pre></p>
<p>will generate this error:<br /><pre>
Error: Event nginx:http__subrequest__start: Invalid userspace probe location (channel channel0, session auto-20190327-105118)
</pre></p> LTTng-tools - Feature #986 (New): Warn when unreasonably short timer values are usedhttps://bugs.lttng.org/issues/9862016-01-05T22:35:59ZJérémie Galarneaujeremie.galarneau@efficios.com
<p>Most timer configuration options are expressed in microseconds which makes it easy for users to specify an unreasonably short intervals of time. These configuration mistakes result in unusually high tracing overhead and may be quite challenging to figure out for inexperienced users.</p>
<p>The lttng client should warn when timers are set to very low values.</p> LTTng-modules - Feature #950 (New): Exclude specific kernel events when enabling all of themhttps://bugs.lttng.org/issues/9502015-10-09T22:55:29ZJulien Desfossezjdesfossez@efficios.com
<p>It would be nice to be able to do that (like in UST):<br />lttng enable-event -k -a --exclude sched_switch</p>
<p>My current workaround:<br />lttng enable-event <del>k $(lttng list -k | grep -v -E "Kernel|---</del>" | grep -v sched_switch | awk '{print $1}' | tr "\n" ",")</p> LTTng-tools - Feature #932 (Confirmed): LTTng client error code semantics are ill-definedhttps://bugs.lttng.org/issues/9322015-09-05T16:34:01ZJérémie Galarneaujeremie.galarneau@efficios.com
<p>The LTTNG (1) man page mentions that</p>
<pre>
On success 0 is returned and a positive value on error. Value of 1 means a command error, 2 an undefined command, 3 a fatal error and 4 a command warning meaning that something went wrong during the command.
Any other value above 10, please refer to <lttng/lttng-error.h> for a detailed list or use lttng_strerror() to get a human readable string of the error code.
</pre>
<p>There are a number of things wrong with this. First of all, asking script authors to check some obscure error header deep in the bowels of their system is questionable, at best.</p>
<p>Second, as remarked by Michael Jeanson, Philippe Proulx, most command-line utilities don't return an error code to indicate a warning. A non-zero error code means an error as occurred.</p>
<p>We will have to go through commands and determine which errors are in fact harmless warnings (e.g. enabling the same event twice), and which ones shall signal an error (e.g. invalid channel configuration).</p>