Project

General

Profile

Actions
Copy link

Bug #1181

open

Deadlock in session daemon and consumer for short lived apps in live per-pid

Added by Francis Deslauriers over 5 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
04/01/2019
Due date:
% Done:

0%

Estimated time:

Description

I encountered a deadlock of the session daemon when tracing short lived UST app in burst. I was not able to reproduce it.
Here is the session configuration:

lttng create allo --live
lttng enable-channel --buffers-pid -s allo --tracefile-size=5M --tracefile-count=4 -u chan1
lttng enable-event -u -a -c chan1
lttng start allo

After configured this session, I ran the following for loop followed by a lttng destroy command. As such:
for i in `seq 1 10`; do
   ../../../lttng-emojis/main & ;
done;
lttng destroy -a

These apps run and exit very quickly. The lttng destroy command never completes .

I attached the source code of the app as well as the backtraces of both lttng session daemon and the lttng consumer daemon.

We gathered so far:
sessiond thread 8: waiting on consumer socket (recv), while holding the consumer socket lock
sessiond thread 11: waiting for socket lock, while holding ust registry lock
sessiond thread 15: waiting for ust registry lock

consumerd thread 4: waiting for stream lock
consumerd thread 7: waiting on consumer metadata socket(recv), while holding stream lock
consumerd thread 8: waiting for stream lock

I was running this lttng-tools branch: https://github.com/PSRCode/lttng-tools/tree/live-per-pid (commit f0e3b9ebe)

Files

Download all files
consumer-bt.txt (6.66 KB) consumer-bt.txt Francis Deslauriers, 04/01/2019 03:20 PM
sessiond-bt.txt (11.6 KB) sessiond-bt.txt Francis Deslauriers, 04/01/2019 03:20 PM
emoji.tgz (1.06 KB) emoji.tgz Francis Deslauriers, 04/01/2019 03:28 PM
relayd-bt-2.txt (6.47 KB) relayd-bt-2.txt Francis Deslauriers, 04/01/2019 04:08 PM
consumerd-bt-2.txt (6.66 KB) consumerd-bt-2.txt Francis Deslauriers, 04/01/2019 04:08 PM
sessiond-bt-2.txt (11.7 KB) sessiond-bt-2.txt Francis Deslauriers, 04/01/2019 04:08 PM
Actions
Copy link
 #1

Updated by Francis Deslauriers over 5 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Francis Deslauriers over 5 years ago

  • Description updated (diff)
Actions
Copy link Download all files
 #3

Updated by Francis Deslauriers over 5 years ago

I was able to reproduce it by using the following loop:

for i in `seq 1 40`; do 
  ./setup-trace.sh ; 
  ../../../lttng-emojis/main &; 
  lttng destroy -a ;
done;

I attached the backtrace of this run.

Actions
Copy link
 #4

Updated by Francis Deslauriers over 5 years ago

  • Description updated (diff)
Actions
Copy link

Also available in: Atom PDF