Project

General

Profile

Actions
Copy link

Bug #574

closed

lttng 2.2.0-rc2: reliable crash

Added by Daniel U. Thibault over 11 years ago. Updated over 11 years ago.

Status:
Resolved
Priority:
High
Assignee:
Mathieu Desnoyers
Target version:
LTTng - 2.3
Start date:
06/25/2013
Due date:
% Done:

0%

Estimated time:

Description

Here is a reliable way of crashing LTTng:

$ sudo -H lttng create sss
Session sss created.
Traces will be written in /root/lttng-traces/sss-20130625-103225
$ sudo -H lttng enable-channel u0 -u --buffers-uid
UST channel u0 enabled for session sss
$ sudo -H lttng enable-channel k0 -k
Kernel channel k0 enabled for session sss
$ sudo -H lttng start
Tracing started for session sss
$ sudo -H lttng stop
Waiting for data availability
Tracing stopped for session sss
$ sudo -H lttng enable-channel u4 -u --buffers-uid
UST channel u4 enabled for session sss
$ sudo -H lttng start
Tracing started for session sss
$ sudo -H lttng enable-channel u6 -u --buffers-uid --subbuf-size 4096
UST channel u6 enabled for session sss
$ sudo -H lttng stop

At which point LTTng hangs and must be killed manually. Once the session daemon is killed, the "Waiting for data availability" message appears, followed by an awful lot of periods.

I'm using:
lttng-modules-2.2.0-rc2-11-5ca7b8a
lttng-tools 2.2.0-rc2-73-51fedb7
lttng-ust-2.2.0-rc2-34-76aca78
userspace-rcu-0.7.6-50-2f49a49

Files

bug-574.txt (27.2 KB) bug-574.txt lttng-sessiond -vvv ---verbose-consumer Daniel U. Thibault, 06/25/2013 01:26 PM
Actions
Copy link
 #1

Updated by David Goulet over 11 years ago

I'm unable to reproduce this issue with the exact same command.

Do you have any applications running during that test ?

Also, can you provide the logs of the session daemon with "-vvv --verbose-consumer". That would be very helpful since I can't reproduce that :S.

Actions
Copy link
 #2

Updated by David Goulet over 11 years ago

  • Status changed from New to Confirmed
  • Assignee set to David Goulet

It has been reproduced on Christian computer. We are hunting this one down!

Actions
Copy link
 #3

Updated by Daniel U. Thibault over 11 years ago

Attached is my capture of the session daemon output.

Actions
Copy link
 #4

Updated by David Goulet over 11 years ago

  • Assignee deleted (David Goulet)
  • Target version changed from 2.2 to 2.3

This affects lttng-modules HEAD master.

lttng-modules-2.2.0-rc2-11-5ca7b8a --> "5ca7b8a" is a commit from master and not the 2-2.0-rc2 tag.

Tagging this for 2.3 since it's an issue added for the next 2.3 release (in master branch).

Actions
Copy link
 #5

Updated by Mathieu Desnoyers over 11 years ago

Simplified script to reproduce:

lttng create sss
sleep 2
lttng enable-channel k0 -k
sleep 2
lttng start
sleep 2
lttng stop
sleep 2
lttng start
sleep 2
lttng stop
sleep 2

It only occurs with master branch of lttng-modules. This is 2.3 material.

Actions
Copy link
 #6

Updated by Mathieu Desnoyers over 11 years ago

  • Assignee set to Julien Desfossez

commit:

commit d83004aa718065e5221d8ef4b99e4302b70b3df9
Author: Julien Desfossez <jdesfossez@efficios.com>
Date:   Thu Jun 6 16:38:48 2013 -0400

    Metadata cache and multiple metadata channels

    Implement a cache for the kernel metadata and a list of metadata
    channels.

    When new metadata is appended, all metadata channels are awakened so
    they can return from poll and get the newly added metadata
    This allows to request the metadata multiple times by creating multiple
    metadata channels (useful for snapshots).

    With this new feature, the poll and get_subbuf ring buffer operations
    are now overridden by lttng-abi for the metadata channels, to check the
    cache before doing these operations.

    Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

is very likely the cause of this issue.

Actions
Copy link
 #7

Updated by Mathieu Desnoyers over 11 years ago

  • Project changed from LTTng-tools to LTTng-modules
  • Status changed from Confirmed to Resolved
  • Assignee changed from Julien Desfossez to Mathieu Desnoyers

Resolved by commit:

commit 9de2c21569afcb6b983e81a2e9b1d0c2b83c98f0
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date:   Wed Jun 26 13:55:31 2013 -0400

    Fix: don't do 0 byte event reservation

    Caused sequence of:

    lttng create
    lttng enable-event -k sched_switch
    lttng start
    lttng stop
    lttng start
    lttng stop

    to hang on waiting for data......

    Issue appeared in

    commit d83004aa718065e5221d8ef4b99e4302b70b3df9
    Author: Julien Desfossez <jdesfossez@efficios.com>
    Date:   Thu Jun 6 16:38:48 2013 -0400

        Metadata cache and multiple metadata channels

    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Actions
Copy link

Also available in: Atom PDF