Actions
Bug #1169
closedPossible race-condition in lttng-ust-statedump.c cleanup code during process exit.
Start date:
09/19/2018
Due date:
% Done:
100%
Estimated time:
Description
We have a short-lived program called 'testcmd' running on a Broadwell x86 processor with 8 logical cores (4 physical)
During cleanup/exit of the process while the system is under load (stress -c 6) we are seeing occasional core-dumps with similar traceback.
After inspecting the code it does not appear that there is locking within lttng-ust-statedump.c for the lttng_ust_statedump_destroy() function when it destroys the list. This patch seems to have resolved the issue for us.
diff --git a/liblttng-ust/lttng-ust-statedump.c b/liblttng-ust/lttng-ust-statedump.c
index efa8a55..1b135de 100644
--- a/liblttng-ust/lttng-ust-statedump.c
+++ b/liblttng-ust/lttng-ust-statedump.c
@@ -630,6 +630,8 @@ void ust_dl_state_destroy(void)
{
unsigned int i;
+ ust_lock_nocheck();
+
for (i = 0; i < UST_DL_STATE_TABLE_SIZE; i++) {
struct cds_hlist_head *head;
struct lttng_ust_dl_node *e, *tmp;
@@ -639,6 +641,8 @@ void ust_dl_state_destroy(void)
free_dl_node(e);
CDS_INIT_HLIST_HEAD(head);
}
+
+ ust_unlock();
}
void lttng_ust_statedump_destroy(void)
--
2.1.1
Backtrace + some additional details are provided below. Please let me know if you need additional information.
#VERSION: This is using the source from the official lttng-ust 2.10.1 tarball.
#https://lttng.org/files/lttng-ust/lttng-ust-2.10.1.tar.bz2
# Here is the test program. Typically by the 40th iteration of the loop the program has crashed and dumped core at least once.
COUNTER=0; while [ 1 ]; do echo "ITER ${COUNTER}"; testcmd show c; ((COUNTER=COUNTER+1)); usleep 500000; done
# After setting the environment variable and re-running the above test, the program no longer crashes after >300 iterations.
export LTTNG_UST_DEBUG=1
### Here is the full backtrace of the threads present at the time of the core dump.
### This has been reproduced several times. As mentioned above, it usually happens by the 40th iteration.
Thread 3 (LWP 22540):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
No locals.
#1 0x000000344e21a246 in futex (val3=0, uaddr2=0x0, timeout=0x0, val=0, op=0, uaddr=<optimized out>) at /usr/include/urcu/futex.h:65
No locals.
#2 futex_async (op=0, val=0, timeout=0x0, uaddr2=0x0, val3=0, uaddr=<optimized out>) at /usr/include/urcu/futex.h:97
No locals.
#3 wait_for_sessiond (sock_info=0x344e4709e0 <local_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1311
No locals.
#4 ust_listener_thread (arg=0x344e4709e0 <local_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1373
sock_info = 0x344e4709e0 <local_apps>
sock = <optimized out>
ret = <optimized out>
prev_connect_failed = 1
has_waited = <optimized out>
timeout = <optimized out>
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#5 0x00000034460073e7 in start_thread (arg=0x7fcff3fde700) at /usr/src/debug/glibc/2.26-r0/git/nptl/pthread_create.c:465
pd = 0x7fcff3fde700
now = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {140531128461056, -6091954114337190402, 140734590213134, 140734590213135, 8396800, 140734590213312, 6118966899948899838, -6116587572263268866},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x0, 0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = <optimized out>
#6 0x00000034458f0bcf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
Thread 2 (LWP 22538):
#0 __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
No locals.
#1 0x000000344587d204 in _int_free (av=0x3445baac20 <main_arena>, p=0x606160, have_lock=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:4273
size = 4192
fb = <optimized out>
nextchunk = <optimized out>
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
__func__ = "_int_free"
#2 0x000000344e22f2a1 in free_dl_node (e=0x606170) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:122
No locals.
#3 ust_dl_state_destroy () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:639
e = 0x606170
tmp = 0x0
i = <optimized out>
#4 lttng_ust_statedump_destroy () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:649
No locals.
#5 0x000000344e218f4b in lttng_ust_cleanup (exiting=1) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1815
No locals.
#6 0x000000344e20f077 in lttng_ust_exit () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1874
ret = <optimized out>
__func__ = "lttng_ust_exit"
#7 0x000000344540f943 in _dl_fini () at /usr/src/debug/glibc/2.26-r0/git/elf/dl-fini.c:235
array = 0x344e469e60
i = <optimized out>
l = 0x7fcff47e4000
maps = 0x7fff5341dfa0
i = <optimized out>
l = <optimized out>
nmaps = <optimized out>
nloaded = <optimized out>
ns = 0
do_audit = <optimized out>
__PRETTY_FUNCTION__ = "_dl_fini"
#8 0x0000003445836608 in __run_exit_handlers (status=0, listp=0x3445baa6f8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at /usr/src/debug/glibc/2.26-r0/git/stdlib/exit.c:83
atfct = <optimized out>
onfct = <optimized out>
cxafct = <optimized out>
f = <optimized out>
#9 0x000000344583665a in __GI_exit (status=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/stdlib/exit.c:105
No locals.
#10 0x0000003445820f11 in __libc_start_main (main=0x400940 <main>, argc=3, argv=0x7fff5341e188, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff5341e178) at /usr/src/debug/glibc/2.26-r0/git/csu/libc-start.c:342
result = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {0, -6091953564717691394, 4197168, 140734590214528, 0, 0, 6092154793035550206, -6116588655236100610},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x7fff5341e1a8, 0x3445626110},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 1396826536
}
}
}
not_first_call = <optimized out>
#11 0x0000000000400b5a in _start () at ../sysdeps/x86_64/start.S:120
No locals.
Thread 1 (LWP 22539):
#0 __GI_raise (sig=sig@entry=6) at /usr/src/debug/glibc/2.26-r0/git/sysdeps/unix/sysv/linux/raise.c:51
set = {
__val = {18446744067266838239, 18446744073709551615 <repeats 13 times>, 18446744067266838271, 0}
}
pid = <optimized out>
tid = <optimized out>
#1 0x0000003445835157 in __GI_abort () at /usr/src/debug/glibc/2.26-r0/git/stdlib/abort.c:90
save_stage = 2
act = {
__sigaction_handler = {
sa_handler = 0xffffffffffffffff,
sa_sigaction = 0xffffffffffffffff
},
sa_mask = {
__val = {18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744067266838271, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744067266838271, 140531136848272, 140531136848560}
},
sa_flags = -193076848,
sa_restorer = 0x1000
}
sigs = {
__val = {32, 0 <repeats 15 times>}
}
#2 0x00000034458751b8 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x34459750fc "%s\n") at /usr/src/debug/glibc/2.26-r0/git/sysdeps/posix/libc_fatal.c:181
ap = {{
gp_offset = 24,
fp_offset = 0,
overflow_arg_area = 0x7fcff47de2c0,
reg_save_area = 0x7fcff47de250
}}
fd = <optimized out>
list = <optimized out>
nlist = <optimized out>
cp = <optimized out>
written = <optimized out>
#3 0x000000344587b33a in malloc_printerr (str=str@entry=0x3445976d30 "double free or corruption (!prev)") at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:5368
No locals.
#4 0x000000344587cffc in _int_free (av=0x3445baac20 <main_arena>, p=0x610940, have_lock=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:4288
size = 4192
fb = <optimized out>
nextchunk = 0x6119a0
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
__func__ = "_int_free"
#5 0x000000344e22eb30 in free_dl_node (e=0x610950) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:122
No locals.
#6 iter_end (data=<optimized out>, ip=<optimized out>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:440
e = 0x610950
#7 lttng_ust_dl_update (ip=0x344e224429 <lttng_handle_pending_statedump+9>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:575
data = {
exec_found = 0,
first = false,
cancel = true
}
ip = 0x344e224429 <lttng_handle_pending_statedump+9>
#8 0x000000344e22f107 in do_baddr_statedump (owner=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:588
No locals.
#9 do_lttng_ust_statedump (owner=owner@entry=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:611
No locals.
#10 0x000000344e224429 in lttng_handle_pending_statedump (owner=owner@entry=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-events.c:871
session = <optimized out>
#11 0x000000344e21aa4d in handle_pending_statedump (sock_info=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:623
ctor_passed = 1
#12 handle_message (lum=0x7fcff47de8b0, sock=<optimized out>, sock_info=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:981
ret = <optimized out>
lur = {
handle = 0,
cmd = 68,
ret_code = 0,
ret_val = 0,
padding = '\000' <repeats 31 times>,
u = {
channel = {
memory_map_size = 0
},
stream = {
memory_map_size = 0
},
version = {
major = 0,
minor = 0,
patchlevel = 0
},
tracepoint = {
name = '\000' <repeats 255 times>,
loglevel = 0,
padding = '\000' <repeats 15 times>
},
padding = '\000' <repeats 31 times>
}
}
ctxstr = "\005", '\000' <repeats 254 times>
len = <optimized out>
ops = <optimized out>
args = {
channel = {
chan_data = 0x1500000014,
wakeup_fd = 5
},
stream = {
shm_fd = 20,
wakeup_fd = 21
},
field_list = {
entry = {
event_name = "\024\000\000\000\025\000\000\000\005", '\000' <repeats 246 times>,
field_name = '\000' <repeats 255 times>,
type = LTTNG_UST_FIELD_OTHER,
loglevel = 0,
nowrite = 0,
padding = '\000' <repeats 283 times>
}
},
app_context = {
ctxname = 0x1500000014 <error: Cannot access memory at address 0x1500000014>
}
}
#13 ust_listener_thread (arg=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1589
len = <optimized out>
lum = {
handle = 0,
cmd = 68,
padding = '\000' <repeats 31 times>,
u = {
channel = {
len = 0,
type = LTTNG_UST_CHAN_PER_CPU,
padding = '\000' <repeats 287 times>,
data = 0x7fcff47dea04 ""
},
stream = {
len = 0,
stream_nr = 0,
padding = '\000' <repeats 287 times>
},
event = {
instrumentation = LTTNG_UST_TRACEPOINT,
name = '\000' <repeats 255 times>,
loglevel_type = LTTNG_UST_LOGLEVEL_ALL,
loglevel = 0,
padding = '\000' <repeats 15 times>,
u = {
padding = '\000' <repeats 287 times>
}
},
context = {
ctx = LTTNG_UST_CONTEXT_VTID,
padding = '\000' <repeats 15 times>,
u = {
perf_counter = {
type = 0,
config = 0,
name = '\000' <repeats 255 times>
},
app_ctx = {
provider_name_len = 0,
ctx_name_len = 0
},
padding = '\000' <repeats 287 times>
}
},
version = {
major = 0,
minor = 0,
patchlevel = 0
},
tracepoint = {
name = '\000' <repeats 255 times>,
loglevel = 0,
padding = '\000' <repeats 15 times>
},
filter = {
data_size = 0,
reloc_offset = 0,
seqnum = 0
},
exclusion = {
count = 0
},
padding = '\000' <repeats 31 times>
}
}
sock_info = 0x344e472a20 <global_apps>
sock = <optimized out>
ret = <optimized out>
prev_connect_failed = 0
has_waited = <optimized out>
timeout = <optimized out>
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#14 0x00000034460073e7 in start_thread (arg=0x7fcff47df700) at /usr/src/debug/glibc/2.26-r0/git/nptl/pthread_create.c:465
pd = 0x7fcff47df700
now = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {140531136853760, -6091954114337190402, 140734590213134, 140734590213135, 8396800, 140734590213312, 6118950406737612286, -6116587572263268866},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x0, 0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = <optimized out>
#15 0x00000034458f0bcf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
(gdb) thread apply all bt full
Thread 3 (LWP 22540):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
No locals.
#1 0x000000344e21a246 in futex (val3=0, uaddr2=0x0, timeout=0x0, val=0, op=0, uaddr=<optimized out>) at /usr/include/urcu/futex.h:65
No locals.
#2 futex_async (op=0, val=0, timeout=0x0, uaddr2=0x0, val3=0, uaddr=<optimized out>) at /usr/include/urcu/futex.h:97
No locals.
#3 wait_for_sessiond (sock_info=0x344e4709e0 <local_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1311
No locals.
#4 ust_listener_thread (arg=0x344e4709e0 <local_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1373
sock_info = 0x344e4709e0 <local_apps>
sock = <optimized out>
ret = <optimized out>
prev_connect_failed = 1
has_waited = <optimized out>
timeout = <optimized out>
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#5 0x00000034460073e7 in start_thread (arg=0x7fcff3fde700) at /usr/src/debug/glibc/2.26-r0/git/nptl/pthread_create.c:465
pd = 0x7fcff3fde700
now = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {140531128461056, -6091954114337190402, 140734590213134, 140734590213135, 8396800, 140734590213312, 6118966899948899838, -6116587572263268866},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x0, 0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = <optimized out>
#6 0x00000034458f0bcf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
Thread 2 (LWP 22538):
#0 __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
No locals.
#1 0x000000344587d204 in _int_free (av=0x3445baac20 <main_arena>, p=0x606160, have_lock=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:4273
size = 4192
fb = <optimized out>
nextchunk = <optimized out>
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
__func__ = "_int_free"
#2 0x000000344e22f2a1 in free_dl_node (e=0x606170) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:122
No locals.
#3 ust_dl_state_destroy () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:639
e = 0x606170
tmp = 0x0
i = <optimized out>
#4 lttng_ust_statedump_destroy () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:649
No locals.
#5 0x000000344e218f4b in lttng_ust_cleanup (exiting=1) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1815
No locals.
#6 0x000000344e20f077 in lttng_ust_exit () at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1874
ret = <optimized out>
__func__ = "lttng_ust_exit"
#7 0x000000344540f943 in _dl_fini () at /usr/src/debug/glibc/2.26-r0/git/elf/dl-fini.c:235
array = 0x344e469e60
i = <optimized out>
l = 0x7fcff47e4000
maps = 0x7fff5341dfa0
i = <optimized out>
l = <optimized out>
nmaps = <optimized out>
nloaded = <optimized out>
ns = 0
do_audit = <optimized out>
__PRETTY_FUNCTION__ = "_dl_fini"
#8 0x0000003445836608 in __run_exit_handlers (status=0, listp=0x3445baa6f8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at /usr/src/debug/glibc/2.26-r0/git/stdlib/exit.c:83
atfct = <optimized out>
onfct = <optimized out>
cxafct = <optimized out>
f = <optimized out>
#9 0x000000344583665a in __GI_exit (status=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/stdlib/exit.c:105
No locals.
#10 0x0000003445820f11 in __libc_start_main (main=0x400940 <main>, argc=3, argv=0x7fff5341e188, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff5341e178) at /usr/src/debug/glibc/2.26-r0/git/csu/libc-start.c:342
result = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {0, -6091953564717691394, 4197168, 140734590214528, 0, 0, 6092154793035550206, -6116588655236100610},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x7fff5341e1a8, 0x3445626110},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 1396826536
}
}
}
not_first_call = <optimized out>
#11 0x0000000000400b5a in _start () at ../sysdeps/x86_64/start.S:120
No locals.
Thread 1 (LWP 22539):
#0 __GI_raise (sig=sig@entry=6) at /usr/src/debug/glibc/2.26-r0/git/sysdeps/unix/sysv/linux/raise.c:51
set = {
__val = {18446744067266838239, 18446744073709551615 <repeats 13 times>, 18446744067266838271, 0}
}
pid = <optimized out>
tid = <optimized out>
#1 0x0000003445835157 in __GI_abort () at /usr/src/debug/glibc/2.26-r0/git/stdlib/abort.c:90
save_stage = 2
act = {
__sigaction_handler = {
sa_handler = 0xffffffffffffffff,
sa_sigaction = 0xffffffffffffffff
},
sa_mask = {
__val = {18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744067266838271, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744067266838271, 140531136848272, 140531136848560}
},
sa_flags = -193076848,
sa_restorer = 0x1000
}
sigs = {
__val = {32, 0 <repeats 15 times>}
}
#2 0x00000034458751b8 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x34459750fc "%s\n") at /usr/src/debug/glibc/2.26-r0/git/sysdeps/posix/libc_fatal.c:181
ap = {{
gp_offset = 24,
fp_offset = 0,
overflow_arg_area = 0x7fcff47de2c0,
reg_save_area = 0x7fcff47de250
}}
fd = <optimized out>
list = <optimized out>
nlist = <optimized out>
cp = <optimized out>
written = <optimized out>
#3 0x000000344587b33a in malloc_printerr (str=str@entry=0x3445976d30 "double free or corruption (!prev)") at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:5368
No locals.
#4 0x000000344587cffc in _int_free (av=0x3445baac20 <main_arena>, p=0x610940, have_lock=<optimized out>) at /usr/src/debug/glibc/2.26-r0/git/malloc/malloc.c:4288
size = 4192
fb = <optimized out>
nextchunk = 0x6119a0
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
__func__ = "_int_free"
#5 0x000000344e22eb30 in free_dl_node (e=0x610950) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:122
No locals.
#6 iter_end (data=<optimized out>, ip=<optimized out>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:440
e = 0x610950
#7 lttng_ust_dl_update (ip=0x344e224429 <lttng_handle_pending_statedump+9>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:575
data = {
exec_found = 0,
first = false,
cancel = true
}
ip = 0x344e224429 <lttng_handle_pending_statedump+9>
#8 0x000000344e22f107 in do_baddr_statedump (owner=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:588
No locals.
#9 do_lttng_ust_statedump (owner=owner@entry=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-statedump.c:611
No locals.
#10 0x000000344e224429 in lttng_handle_pending_statedump (owner=owner@entry=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-events.c:871
session = <optimized out>
#11 0x000000344e21aa4d in handle_pending_statedump (sock_info=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:623
ctor_passed = 1
#12 handle_message (lum=0x7fcff47de8b0, sock=<optimized out>, sock_info=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:981
ret = <optimized out>
lur = {
handle = 0,
cmd = 68,
ret_code = 0,
ret_val = 0,
padding = '\000' <repeats 31 times>,
u = {
channel = {
memory_map_size = 0
},
stream = {
memory_map_size = 0
},
version = {
major = 0,
minor = 0,
patchlevel = 0
},
tracepoint = {
name = '\000' <repeats 255 times>,
loglevel = 0,
padding = '\000' <repeats 15 times>
},
padding = '\000' <repeats 31 times>
}
}
ctxstr = "\005", '\000' <repeats 254 times>
len = <optimized out>
ops = <optimized out>
args = {
channel = {
chan_data = 0x1500000014,
wakeup_fd = 5
},
stream = {
shm_fd = 20,
wakeup_fd = 21
},
field_list = {
entry = {
event_name = "\024\000\000\000\025\000\000\000\005", '\000' <repeats 246 times>,
field_name = '\000' <repeats 255 times>,
type = LTTNG_UST_FIELD_OTHER,
loglevel = 0,
nowrite = 0,
padding = '\000' <repeats 283 times>
}
},
app_context = {
ctxname = 0x1500000014 <error: Cannot access memory at address 0x1500000014>
}
}
#13 ust_listener_thread (arg=0x344e472a20 <global_apps>) at /usr/src/debug/lttng-ust/2_2.10.1-r0/lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1589
len = <optimized out>
lum = {
handle = 0,
cmd = 68,
padding = '\000' <repeats 31 times>,
u = {
channel = {
len = 0,
type = LTTNG_UST_CHAN_PER_CPU,
padding = '\000' <repeats 287 times>,
data = 0x7fcff47dea04 ""
},
stream = {
len = 0,
stream_nr = 0,
padding = '\000' <repeats 287 times>
},
event = {
instrumentation = LTTNG_UST_TRACEPOINT,
name = '\000' <repeats 255 times>,
loglevel_type = LTTNG_UST_LOGLEVEL_ALL,
loglevel = 0,
padding = '\000' <repeats 15 times>,
u = {
padding = '\000' <repeats 287 times>
}
},
context = {
ctx = LTTNG_UST_CONTEXT_VTID,
padding = '\000' <repeats 15 times>,
u = {
perf_counter = {
type = 0,
config = 0,
name = '\000' <repeats 255 times>
},
app_ctx = {
provider_name_len = 0,
ctx_name_len = 0
},
padding = '\000' <repeats 287 times>
}
},
version = {
major = 0,
minor = 0,
patchlevel = 0
},
tracepoint = {
name = '\000' <repeats 255 times>,
loglevel = 0,
padding = '\000' <repeats 15 times>
},
filter = {
data_size = 0,
reloc_offset = 0,
seqnum = 0
},
exclusion = {
count = 0
},
padding = '\000' <repeats 31 times>
}
}
sock_info = 0x344e472a20 <global_apps>
sock = <optimized out>
ret = <optimized out>
prev_connect_failed = 0
has_waited = <optimized out>
timeout = <optimized out>
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#14 0x00000034460073e7 in start_thread (arg=0x7fcff47df700) at /usr/src/debug/glibc/2.26-r0/git/nptl/pthread_create.c:465
pd = 0x7fcff47df700
now = <optimized out>
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {140531136853760, -6091954114337190402, 140734590213134, 140734590213135, 8396800, 140734590213312, 6118950406737612286, -6116587572263268866},
mask_was_saved = 0
}},
priv = {
pad = {0x0, 0x0, 0x0, 0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = <optimized out>
#15 0x00000034458f0bcf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
(gdb)
### This is the output of a single execution of the above command with LTTNG_UST_DEBUG environment variable set. ### (Program does not crash and exits normally.) root@hal-dnx:/# testcmd show c liblttng_ust_tracepoint[6722/6722]: Your compiler treats weak symbols with hidden visibility for integer objects as SAME address between compile units part of the same module. (in check_weak_hidden() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:916) liblttng_ust_tracepoint[6722/6722]: Your compiler treats weak symbols with hidden visibility for pointer objects as SAME address between compile units part of the same module. (in check_weak_hidden() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:920) liblttng_ust_tracepoint[6722/6722]: Your compiler treats weak symbols with hidden visibility for 24-byte structure objects as SAME address between compile units part of the same module. (in check_weak_hidden() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:924) liblttng_ust_tracepoint[6722/6722]: just registered a tracepoints section from 0x344e4751e0 and having 25 tracepoints (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:867) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_statedump:end (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_statedump:debug_link (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_statedump:build_id (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_statedump:bin_info (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_statedump:start (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_lib:unload (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_lib:debug_link (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_lib:build_id (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_lib:load (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracef:event (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_LINE (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_FUNCTION (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_UNIT (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_MODULE (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_PROCESS (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_PROGRAM (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_DEBUG_SYSTEM (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_INFO (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_NOTICE (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_WARNING (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_ERR (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_CRIT (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_ALERT (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) liblttng_ust_tracepoint[6722/6722]: registered tracepoint: lttng_ust_tracelog:TRACE_EMERG (in tracepoint_register_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:872) libust[6722/6722]: Provider "lttng_ust_statedump" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: adding probe lttng_ust_statedump containing 5 events to lazy registration list (in lttng_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:219) libust[6722/6722]: LTT : ltt ring buffer client "relay-metadata-mmap" init (in lttng_ring_buffer_metadata_client_init() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-metadata-client.h:350) libust[6722/6722]: LTT : ltt ring buffer client "relay-overwrite-mmap" init (in lttng_ring_buffer_client_overwrite_init() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:825) libust[6722/6722]: LTT : ltt ring buffer client "relay-overwrite-rt-mmap" init (in lttng_ring_buffer_client_overwrite_rt_init() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:825) libust[6722/6722]: LTT : ltt ring buffer client "relay-discard-mmap" init (in lttng_ring_buffer_client_discard_init() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:825) libust[6722/6722]: LTT : ltt ring buffer client "relay-discard-rt-mmap" init (in lttng_ring_buffer_client_discard_rt_init() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:825) libust[6722/6724]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1424) libust[6722/6724]: Waiting for local apps sessiond (in wait_for_sessiond() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:1306) libust[6722/6723]: Message Received "Get Tracer Version" (65), Handle "root" (0) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Session" (64), Handle "root" (0) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 1 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Channel" (81), Handle "session" (1) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: channel data received (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:811) libust[6722/6723]: Return value: 2 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Stream" (96), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 3 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (3) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 4 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (4) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 5 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (5) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 6 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (6) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 7 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (7) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 8 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (8) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 9 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (9) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 10 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (10) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 11 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (11) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 12 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (12) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 13 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (13) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 14 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (14) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 15 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (15) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 16 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (16) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 17 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (17) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 18 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (18) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 19 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (19) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 20 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (20) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 21 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (21) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 22 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (22) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 23 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (23) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Create Event" (97), Handle "channel" (2) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 24 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "enabler" (24) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Enable" (128), Handle "session" (1) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Sent register channel notification: chan_id 0, header_type 1 (in ustcomm_register_channel() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1511) libust[6722/6723]: just registered probe lttng_ust_statedump containing 5 events (in lttng_lazy_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:116) liblttng_ust_tracepoint[6722/6723]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:702) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Wait for Quiescent State" (67), Handle "root" (0) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6723]: message successfully sent (in send_reply() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:571) libust[6722/6723]: Message Received "Registration Done" (68), Handle "root" (0) (in print_cmd() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:430) libust[6722/6722]: Provider "lttng_ust_lib" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6723]: Return value: 0 (in handle_message() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ust-comm.c:972) libust[6722/6722]: adding probe lttng_ust_lib containing 4 events to lazy registration list (in lttng_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:219) libust[6722/6722]: just registered probe lttng_ust_lib containing 4 events (in lttng_lazy_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:116) liblttng_ust_tracepoint[6722/6722]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:702) libust[6722/6722]: Provider "lttng_ust_tracef" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: adding probe lttng_ust_tracef containing 1 events to lazy registration list (in lttng_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:219) libust[6722/6722]: just registered probe lttng_ust_tracef containing 1 events (in lttng_lazy_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:116) liblttng_ust_tracepoint[6722/6722]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:702) libust[6722/6722]: Provider "lttng_ust_tracelog" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: adding probe lttng_ust_tracelog containing 15 events to lazy registration list (in lttng_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:219) libust[6722/6722]: just registered probe lttng_ust_tracelog containing 15 events (in lttng_lazy_probe_register() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:116) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_EMERG": ret_code 0, event_id 0 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_ALERT": ret_code 0, event_id 1 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_CRIT": ret_code 0, event_id 2 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_ERR": ret_code 0, event_id 3 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_WARNING": ret_code 0, event_id 4 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_NOTICE": ret_code 0, event_id 5 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) libust[6722/6722]: Sent register event notification for name "lttng_ust_tracelog:TRACE_INFO": ret_code 0, event_id 6 (in ustcomm_register_event() at ../../lttng-ust-2.10.1/liblttng-ust-comm/lttng-ust-comm.c:1293) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_INFO. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_NOTICE. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_WARNING. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_ERR. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_CRIT. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_ALERT. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Registering probe to tracepoint lttng_ust_tracelog:TRACE_EMERG. Queuing release. (in __tracepoint_probe_register_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:611) liblttng_ust_tracepoint[6722/6722]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:702) libust[6722/6722]: Provider "lttng_ust_tracelog" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: just unregistered probe lttng_ust_tracelog (in lttng_probe_unregister() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:250) libust[6722/6722]: Provider "lttng_ust_tracef" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: just unregistered probe lttng_ust_tracef (in lttng_probe_unregister() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:250) libust[6722/6722]: Provider "lttng_ust_lib" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: just unregistered probe lttng_ust_lib (in lttng_probe_unregister() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:250) |liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_INFO. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_NOTICE. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_WARNING. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_ERR. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_CRIT. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_ALERT. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Un-registering probe from tracepoint lttng_ust_tracelog:TRACE_EMERG. Queuing release. (in __tracepoint_probe_unregister_queue_release() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:682) liblttng_ust_tracepoint[6722/6722]: Release queue of unregistered tracepoint probes. (in __tracepoint_probe_prune_release_queue() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:702) libust[6722/6722]: LTT : ltt ring buffer client "relay-discard-rt-mmap" exit (in lttng_ring_buffer_client_discard_rt_exit() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:832) libust[6722/6722]: LTT : ltt ring buffer client "relay-discard-mmap" exit (in lttng_ring_buffer_client_discard_exit() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:832) libust[6722/6722]: LTT : ltt ring buffer client "relay-overwrite-rt-mmap" exit (in lttng_ring_buffer_client_overwrite_rt_exit() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:832) libust[6722/6722]: LTT : ltt ring buffer client "relay-overwrite-mmap" exit (in lttng_ring_buffer_client_overwrite_exit() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-client.h:832) libust[6722/6722]: LTT : ltt ring buffer client "relay-metadata-mmap" exit (in lttng_ring_buffer_metadata_client_exit() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-ring-buffer-metadata-client.h:357) libust[6722/6722]: Provider "lttng_ust_statedump" accepted, version 1.0 is compatible with LTTng UST provider version 1.0. (in check_provider_version() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:175) libust[6722/6722]: just unregistered probe lttng_ust_statedump (in lttng_probe_unregister() at ../../lttng-ust-2.10.1/liblttng-ust/lttng-probes.c:250) liblttng_ust_tracepoint[6722/6722]: just unregistered a tracepoints section from 0x344e4751e0 (in tracepoint_unregister_lib() at ../../lttng-ust-2.10.1/liblttng-ust/tracepoint.c:897) root@hal-dnx:/# root@hal-dnx:/#
Files
Updated by Jonathan Rajotte Julien over 5 years ago
- Description updated (diff)
- Status changed from New to Feedback
- Assignee set to Mathieu Desnoyers
Mathieu can you take a look at this?
Thanks
Updated by Mathieu Desnoyers over 5 years ago
- File 0001-Fix-race-between-statedump-and-library-destructor.patch 0001-Fix-race-between-statedump-and-library-destructor.patch added
- Status changed from Feedback to Confirmed
Can you try with the attached patch ?
Updated by Mathieu Desnoyers over 5 years ago
- Status changed from Confirmed to Resolved
- % Done changed from 0 to 100
Applied in changeset lttng-ust|2eb235ecd28828cef90ff371a96cd532e92ba9f6.
Updated by Rob Maher over 5 years ago
Mathieu Desnoyers wrote:
Can you try with the attached patch ?
After reproducing the original issue I downloaded the patch from the git repo and applied it.
Still no crashes after ~25000 iterations.
Thank you for the fix!
Actions