Bug #1036
closedChannel creation stops working and returns "Command error" without apparent reason after a few iterations of session/channel create+destroy
0%
Description
Environment: Chroot on an ARM phone running Ubuntu Touch (15.04-based)
Tools:
lttng-ust 2.8.1 built from stable git branch
lttng-tools 2.8.0 built from stable git branch
My workflow is the following:
1) setup session and channel
lttng create
lttng enable-channel --userspace --num-subbuf 8 --subbuf-size 8M big-channel
lttng enable-event -ua --channel big-channel
lttng add-context -u -t vpid -t vtid -t procname -t ip
lttng start
2) run the app in a separate terminal
3) lttng stop + lttng destroy
4) Pull the trace, analyze etc
5) Rinse and repeat
What happens is that at one point (it seems when I wait more than a few minutes between the stop and the destroy commands)
the channel creation command starts failing.
The only way to fix it seem to be rebooting the device.
An example of sessiond log:
DEBUG2 - 13:12:59.823516 [25414/25420]: UST app session created successfully with handle 1 (in create_ust_app_session() at ust-app.c:2246)
DEBUG3 - 13:12:59.823659 [25414/25420]: Created hashtable size 4 at 0xaf0213c8 of type 1 (in lttng_ht_new() at hashtable.c:145)
DEBUG3 - 13:12:59.823745 [25414/25420]: Created hashtable size 4 at 0xaf026bd0 of type 0 (in lttng_ht_new() at hashtable.c:145)
DEBUG3 - 13:12:59.823819 [25414/25420]: UST app channel big-channel allocated (in alloc_ust_app_channel() at ust-app.c:1051)
DEBUG2 - 13:12:59.823888 [25414/25420]: UST app shadow copy of channel big-channel started (in shadow_copy_channel() at ust-app.c:1794)
DEBUG3 - 13:12:59.823966 [25414/25420]: UST app shadow copy of channel big-channel done (in shadow_copy_channel() at ust-app.c:1845)
DEBUG1 - 13:12:59.824041 [25414/25420]: UST app creating channel big-channel with per UID buffers (in create_channel_per_uid() at ust-app.c:2860)
DEBUG3 - 13:12:59.824113 [25414/25420]: Buffer registry per UID find id: 0, ABI: 32, uid: 0 (in buffer_reg_uid_find() at buffer-registry.c:203)
DEBUG2 - 13:12:59.824190 [25414/25420]: UST app creating buffer registry channel for big-channel (in create_buffer_reg_channel() at ust-app.c:2705)
DEBUG3 - 13:12:59.824271 [25414/25420]: Buffer registry channel create with key: 0 (in buffer_reg_channel_create() at buffer-registry.c:380)
DEBUG3 - 13:12:59.824371 [25414/25420]: Created hashtable size 4 at 0xaf021980 of type 0 (in lttng_ht_new() at hashtable.c:145)
DEBUG2 - 13:12:59.824457 [25414/25420]: Asking UST consumer for channel (in ask_channel_creation() at ust-consumer.c:120)
DEBUG3 - 13:12:59.824537 [25414/25420]: mkdir() recursive /home/phablet/lttng-traces/auto-20160704-131253/ust/uid/0/32-bit with mode 504 for uid 32011 and gid 32011 (in run_as_mkdir_recursive() at runas.c:470)
DEBUG1 - 13:12:59.824622 [25414/25420]: Using run_as worker (in run_as() at runas.c:451)
DEBUG3 - 13:12:59.825649 [25414/25420]: UST registry channel finding key 0 (in ust_registry_channel_find() at ust-registry.c:784)
Error: Error creating UST channel "big-channel" on the consumer daemon
An example of LTTNG_UST_DEBUG log (from a different session, sorry)
http://pastebin.ubuntu.com/18448735/
I made sure any session shown by lttng list was destroyed, so lttng list shows nothing
lsof output of consumerd http://pastebin.ubuntu.com/18459042/