Project

General

Profile

Actions
Copy link

Bug #687

closed

Crashing lttng-sessiond with enable-event --function

Added by Daniel U. Thibault about 11 years ago. Updated almost 3 years ago.

Status:
Invalid
Priority:
Low
Assignee:
Mathieu Desnoyers
Target version:
-
Start date:
11/19/2013
Due date:
% Done:

0%

Estimated time:

Description

I managed to crash lttng-sessiond while fooling around with kretprobe events.

First I stopped the lttng-sessiond service and started a verbose root session manager to try and get the crash log (attached).

Then I did this:

$ lttng create krpso
Session krpso created.
Traces will be written in /home/daniel/lttng-traces/krpso-20131119-095940
$ lttng enable-event -k krpso --function sys_open
kernel event krpso created in channel channel0

Having previously run the above tracing session and obtained this babeltrace:

timestamp = 09:53:54.443925388, delta = +?.?????????, trace = /home/daniel/lttng-traces/krpso-20131119-095319/kernel, trace:hostname = sds-dut-vb, trace:domain = kernel, name = krpso_entry, stream.packet.context = { cpu_id = 0 }, event.fields = { ip = 0xFFFFFFFF81176A70, parent_ip = 0xFFFFFFFF81662142 }
timestamp = 09:53:54.443945296, delta = +0.000019908, trace = /home/daniel/lttng-traces/krpso-20131119-095319/kernel, trace:hostname = sds-dut-vb, trace:domain = kernel, name = krpso_return, stream.packet.context = { cpu_id = 0 }, event.fields = { ip = 0xFFFFFFFF81176A70, parent_ip = 0xFFFFFFFF81662142 }

I knew the absolute address of whatever called sys_open, so I continue with this:

$ lttng enable-event -k krpso_parent --function 0xffffffff81662142
kernel event krpso_parent created in channel channel0

At which point the session manager just dies, with no message whatsoever.

If I instead do the equivalent symbol+offset command:

$ lttng enable-event -k krpso_parent --function sys_open+0x4EB6D2
k event k created in channel c
Erreur du bus (core dumped)

Or this (just in case this is related to bug #633):

$ lttng enable-event -k krpso_parent --function sys_open+5158610
k event k created in channel c
Erreur du bus (core dumped)

Files

Download all files
CrashLog.txt (20.6 KB) CrashLog.txt Crash log of lttng-sessiond Daniel U. Thibault, 11/19/2013 10:34 AM
sessiond.log (42.4 KB) sessiond.log Daniel U. Thibault, 02/25/2014 09:02 AM
Actions
Copy link
 #1

Updated by David Goulet almost 11 years ago

  • Status changed from New to Feedback

I'm unable to reproduce this with latest 2.3... maybe you can provide the backtrace of the core dump (gdb> bt full).

In verbose and not in verbose, can't make it crash :S

Actions
Copy link
 #2

Updated by David Goulet almost 11 years ago

Any update on that before we release stable-2.4 ?

Actions
Copy link
 #3

Updated by Daniel U. Thibault almost 11 years ago

Just tried again on a freshly booted virtual machine. Crashed as expected.

lttng-modules-2.3.0-1-78c8710
lttng-tools-2.3.0-1-c9dc128
lttng-ust-2.3.0-1-9f00ce3
userspace-rcu-0.8.0-1-ca617ca
Ubuntu 12.04.4 LTS running 3.9.3 kernel

On one console:

$ sudo service lttng-sessiond stop
lttng-sessiond stop/waiting
$ sudo lttng-sessiond -vvv &> ~/lttng-traces/sessiond.log

sessiond.log attached.

On another console:

$ lttng create krpso
Session krpso created.
Traces will be written in /home/daniel/lttng-traces/krpso-20140225-084038
$ lttng enable-event -k krpso --function sys_open
kernel event krpso created in channel channel0
$ lttng start
Tracing started for session krpso
[run gedit, open a document]
$ lttng stop
Waiting for data availability.
Tracing stopped for session krpso
$ lttng destroy
Session krpso destroyed
$ babeltrace ~/lttng-traces/krpso-20140225-084038/ -w ~/lttng-traces/krpso_na_fa -n all -f all
[krpso_na_fa shows:
... name = krpso_entry, [...] event.fields = { ip = 0xFFFFFFFF81188030, parent_ip = 0xFFFFFFFF816B2159 ]
$ lttng create krpso2
Session krpso2 created.
Traces will be written in /home/daniel/lttng-traces/krpso2-20140225-084513
$ lttng enable-event -k krpso --function sys_open
kernel event krpso created in channel channel0
$ lttng enable-event -k krpso_parent --function 0xffffffff816b2159
kernel event krpso_parent created in channel channel0
[lttng-sessiond crashes]
$ gdb
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying" 
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>.
(gdb) bt full
No stack.
(gdb)

I'm afraid I'm not familiar with gdb, as the above shows. How do I get a backtrace of a core dump?

I suspect the crash is instead due to the hex value not being read correctly (bug #633).

Actions
Copy link
 #4

Updated by David Goulet almost 11 years ago

  • Status changed from Feedback to Confirmed
  • Assignee set to David Goulet
  • Priority changed from Normal to High
Actions
Copy link
 #5

Updated by Mathieu Desnoyers almost 11 years ago

  • Project changed from LTTng-tools to LTTng-modules
Actions
Copy link
 #6

Updated by Mathieu Desnoyers almost 11 years ago

  • Assignee changed from David Goulet to Mathieu Desnoyers

This bug is a missing check within the Linux kernel kretprobe instrumentation infrastructure. Reported to the Linux kernel maintainers.

Actions
Copy link
 #7

Updated by Mathieu Desnoyers almost 11 years ago

  • Status changed from Confirmed to Feedback

LKML post reporting the issue:

https://lkml.org/lkml/2014/2/25/471

Let's await feedback from this thread.

Actions
Copy link
 #8

Updated by Christian Babeux about 10 years ago

  • Priority changed from High to Low
  • Target version deleted (2.3)

This is a kernel bug. We are still waiting on feedback from the Linux community.

Actions
Copy link
 #9

Updated by Michael Jeanson almost 3 years ago

  • Status changed from Feedback to Invalid
Actions
Copy link

Also available in: Atom PDF