SC-1:/storage/no-backup/coremw/var/log/saflog # pkill -9 lttng-sessiond; usleep 90000; lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 & [7] 1484 SC-1:/storage/no-backup/coremw/var/log/saflog # sleep 2;/cluster/temp/eusgarc/test_once DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:3692] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:3940] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:3942] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:3995] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:3996] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:3997] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4006] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4008] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4017] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4019] DEBUG3: Created hashtable size 4 at 0x64a0a0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64a5c0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:3734] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:3734] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:3734] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:3826] DEBUG3: Session daemon client socket 8 and application socket 9 created [in init_daemon_socket() at main.c:3596] Warning: No tracing group detected DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:270] DEBUG1: Command subsystem initialized [in cmd_init() at cmd.c:2423] DEBUG1: [thread] Manage health check started [in thread_manage_health() at main.c:2953] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3147] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1378] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1300] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:496] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1107] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1310] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1135] DEBUG1: Got the wait shm fd 17 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:63] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] Wed Dec 19 10:43:13 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 1484 26613 0 10:43 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:43:13 EST 2012 Creating Profile #################### Wed Dec 19 10:43:13 EST 2012 6441 10:43:13 12/19/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 6442 10:43:13 12/19/2012 IN TraceC "DBG: newvalue size=23 6443 10:43:13 12/19/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProf 6444 10:43:13 12/19/2012 IN TraceC "DBG: traceExpressionsValidator was called for traceExpression with size = 1. 6445 10:43:13 12/19/2012 IN TraceC "DBG: conditionValidator: About to create a temporary session _TraceCValidator_sKNs40 in path /tmp/_TraceCValidator_sKNs40 - Invoke lttng_create_session 6446 10:43:13 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_create_handle for temporary session _TraceCValidator_sKNs40. 6447 10:43:13 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_set_default_handle_attr for temporary session _TraceCValidator_sKNs40 Dec 19 10:43:13 SC-1 osafimmnd[6451]: Ccb 519 COMMITTED (TraceCLI) Dec 19 10:43:13 PL-3 osafimmnd[4905]: Ccb 519 COMMITTED (TraceCLI) Dec 19 10:43:13 SC-2 osafimmnd[5121]: Ccb 519 COMMITTED (TraceCLI) Dec 19 10:43:13 PL-4 osafimmnd[4935]: Ccb 519 COMMITTED (TraceCLI) Wed Dec 19 10:43:13 EST 2012 Wait 1 sec... Dec 19 10:43:13 SC-1 osafimmnd[6451]: Ccb 520 COMMITTED (TraceC) Dec 19 10:43:13 PL-3 osafimmnd[4905]: Ccb 520 COMMITTED (TraceC) Dec 19 10:43:13 PL-4 osafimmnd[4935]: Ccb 520 COMMITTED (TraceC) Dec 19 10:43:13 SC-2 osafimmnd[5121]: Ccb 520 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2204] DEBUG1: Waiting for 1 URIs from client ... [in process_client_msg() at main.c:2711] DEBUG2: Trying to find session by name _TraceCValidator_sKNs40 [in session_find_by_name() at session.c:122] DEBUG1: Tracing session _TraceCValidator_sKNs40 created in (null) with ID 0 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name _TraceCValidator_sKNs40 [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x655b10 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting trace directory path from URI to /tmp/_TraceCValidator_sKNs40 [in add_uri_to_consumer() at cmd.c:407] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2204] DEBUG1: Getting session _TraceCValidator_sKNs40 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name _TraceCValidator_sKNs40 [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2091] DEBUG3: Created hashtable size 4 at 0x64e960 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64eea0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64f3e0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65b0b0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:239] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2043] DEBUG3: Created hashtable size 4 at 0x65b0b0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir /ust [in copy_session_consumer() at main.c:2061] DEBUG1: Spawning consumerd [in spawn_consumerd() at main.c:1722] DEBUG2: Consumer pid 1503 [in start_consumerd() at main.c:1903] DEBUG2: Spawning consumer control thread [in start_consumerd() at main.c:1906] DEBUG1: [thread] Manage consumer started [in thread_manage_consumer() at main.c:856] DEBUG1: Using 64-bit UST consumer at: /usr/lib64/lttng/libexec/lttng-consumerd [in spawn_consumerd() at main.c:1796] DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:270] DEBUG3: Created hashtable size 4 at 0x627030 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627570 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627ab0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627ff0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Connecting to error socket /var/run/lttng/ustconsumerd64/error [in main() at lttng-consumerd.c:358] DEBUG3: Created hashtable size 4 at 0x628830 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Thread metadata poll started [in consumer_thread_metadata_poll() at consumer.c:2092] DEBUG3: Created hashtable size 4 at 0x628d70 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Creating command socket /var/run/lttng/ustconsumerd64/command [in consumer_thread_sessiond_poll() at consumer.c:2545] DEBUG1: Metadata main loop started [in consumer_thread_metadata_poll() at consumer.c:2107] DEBUG1: Metadata poll wait with 1 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: polling on 1 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Sending ready command to lttng-sessiond [in consumer_thread_sessiond_poll() at consumer.c:2558] DEBUG1: Connection on client_socket [in consumer_thread_sessiond_poll() at consumeDEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:179] DEBUG1: Enabling channel channel0 for session _TraceCValidator_sKNs40 [in cmd_enable_channel() at cmd.c:823] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:141] DEBUG3: Created hashtable size 4 at 0x65ca40 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65cf80 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:303] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:277] DEBUG2: UST app adding channel channel0 to global domain for session id 0 [in ust_app_create_channel_glb() at ust-app.c:2043] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:302] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2204] DEBUG1: Getting session _TraceCValidator_sKNs40 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name _TraceCValidator_sKNs40 [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:700] DEBUG1: Destroy all UST traces [in ust_app_destroy_trace_all() at ust-app.c:2570] DEBUG2: Trace UST destroy session 0 [in trace_ust_destroy_session() at trace-ust.c:668] DEBUG1: Destroying session _TraceCValidator_sKNs40 [in session_destroy() at session.c:149] DEBUG2: Trace destroy UST channel channel0 [in trace_ust_destroy_channel() at trace-ust.c:551] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] 6448 10:43:13 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_destroy_session for temporary session _TraceCValidator_sKNs40 6449 10:43:13 12/19/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_sKNs40" 6450 10:43:13 12/19/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_sKNs40" Deleted! 6451 10:43:13 12/19/2012 IN TraceC "DBG: conditionValidator: Reply recieved from lttng_destroy_session for temporary session _TraceCValidator_sKNs40. 6452 10:43:13 12/19/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProf,traceMId=1 Wed Dec 19 10:43:14 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 1484 26613 0 10:43 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 1503 1484 0 10:43 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:43:14 EST 2012 Activating Profile #################### Wed Dec 19 10:43:14 EST 2012 Dec 19 10:43:14 SC-1 osafimmnd[6451]: Ccb 522 COMMITTED (TraceC) Dec 19 10:43:14 PL-3 osafimmnd[4905]: Ccb 522 COMMITTED (TraceC) Dec 19 10:43:14 PL-4 osafimmnd[4935]: Ccb 522 COMMITTED (TraceC) Dec 19 10:43:14 SC-2 osafimmnd[5121]: Ccb 522 COMMITTED (TraceC) Dec 19 10:43:14 SC-1 osafimmnd[6451]: Ccb 523 COMMITTED (TraceC) Dec 19 10:43:14 PL-3 osafimmnd[4905]: Ccb 523 COMMITTED (TraceC) Dec 19 10:43:14 PL-4 osafimmnd[4935]: Ccb 523 COMMITTED (TraceC) Dec 19 10:43:14 SC-2 osafimmnd[5121]: Ccb 523 COMMITTED (TraceC) Dec 19 10:43:14 SC-1 osafimmnd[6451]: Ccb 524 COMMITTED (TraceC) Dec 19 10:43:14 PL-3 osafimmnd[4905]: Ccb 524 COMMITTED (TraceC) Dec 19 10:43:14 PL-4 osafimmnd[4935]: Ccb 524 COMMITTED (TraceC) Dec 19 10:43:14 SC-2 osafimmnd[5121]: Ccb 524 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2204] DEBUG1: Waiting for 2 URIs from client ... [in process_client_msg() at main.c:2711] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Tracing session TestProf_1 created in (null) with ID 1 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x65c900 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer control URI set with port 59383 [in consumer_set_network_uri() at consumer.c:420] DEBUG3: Consumer set network uri subdir path SC-1/./ [in consumer_set_network_uri() at consumer.c:472] DEBUG3: Append domain trace name to subdir SC-1/./ [in add_uri_to_consumer() at cmd.c:402] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer data URI set with port 50886 [in consumer_set_network_uri() at consumer.c:429] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2091] DEBUG3: Created hashtable size 4 at 0x65ce40 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x659c10 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65a150 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65a670 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:239] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2043] DEBUG3: Created hashtable size 4 at 0x65a670 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir SC-1/.//ust [in copy_session_consumer() at main.c:2061] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:179] DEBUG1: Enabling channel channel0 for session TestProf_1 [in cmd_enable_channel() at cmd.c:823] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:141] DEBUG3: Created hashtable size 4 at 0x65abe0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65b120 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:303] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:277] DEBUG2: UST app adding channel channel0 to global domain for session id 1 [in ust_app_create_channel_glb() at ust-app.c:2043] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:302] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 6 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:179] DEBUG2: Trace UST channel channel0 found by name [in trace_ust_find_channel_by_name() at trace-ust.c:136] DEBUG2: Trace UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA NOT found [in trace_ust_find_event() at trace-ust.c:174] DEBUG2: Trace UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, loglevel (1,10) created [in trace_ust_create_event() at trace-ust.c:379] DEBUG1: UST app creating event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA for all apps for session id 1 [in ust_app_create_event_glb() at ust-app.c:2171] DEBUG1: Event UST com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created in channel channel0 [in event_ust_enable_tracepoint() at event.c:476] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 16 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG2: Setting relayd for session TestProf_1 [in setup_relayd() at cmd.c:644] DEBUG3: Relayd connect ... [in relayd_connect() at relayd.c:315] DEBUG3: Creating relayd stream socket from URI [in create_connect_relayd() at cmd.c:492] DEBUG1: Relayd version check for major.minor 2.1 [in relayd_version_check() at relayd.c:219] DEBUG3: Relayd sending command 5 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 8 [in recv_reply() at relayd.c:90] DEBUG2: Relayd version is compatible [in relayd_version_check() at relayd.c:254] DEBUG3: Sending relayd sock info to consumer on 20 [in consumer_send_relayd_socket() at consumer.c:690] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Consumer adding relayd socket (idx: 2) [in consumer_add_relayd_socket() at consumer.c:2731] DEBUG3: Sending relayd socket file descriptor to consumer [in consumer_send_relayd_socket() at consumer.c:702] Dec 19 10:43:15 SC-1 osafimmnd[6451]: Ccb 525 COMMITTED (TraceC) Dec 19 10:43:15 PL-3 osafimmnd[4905]: Ccb 525 COMMITTED (TraceC) Dec 19 10:43:15 PL-4 osafimmnd[4935]: Ccb 525 COMMITTED (TraceC) Dec 19 10:43:15 SC-2 osafimmnd[5121]: Ccb 525 COMMITTED (TraceC) 6453 10:43:14 12/19/2012 IN TraceC "DBG: Activating 6454 10:43:14 12/19/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProf_1,traceMId=1 as a runtime object 6455 10:43:14 12/19/2012 NO TraceC "Activating Session TestProf_1. 6456 10:43:14 12/19/2012 IN TraceC "DBG: Directory /storage/no-backup/tracecc-apr9010500//TestProf_1 was created successfully! 6457 10:43:14 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6458 10:43:14 12/19/2012 IN TraceC "DBG: performActivate: Starting RelayD for session TestProf_1. 6459 10:43:14 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6460 10:43:14 12/19/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 59383 6461 10:43:14 12/19/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 50886 6462 10:43:14 12/19/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 6463 10:43:14 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6464 10:43:14 12/19/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProf_1" pid: 1534 bound on 192.168.0.1:59383:50886 6465 10:43:14 12/19/2012 IN TraceC "DBG: performActivate: Writing session into file 6466 10:43:14 12/19/2012 IN TraceC "DBG: Done Activating 6467 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProf_1 is being added to parent directory /storage/no-backup/tracecc-apr9010500 6468 10:43:14 12/19/2012 IN TraceC "DBG: RelayD has been launch for TestProf_1 session after 1 tries to connect to free ports. 6469 10:43:14 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6470 10:43:14 12/19/2012 IN TraceC "DBG: performActivateAfterRelayD: Sending commands to the trace_ps. 6471 10:43:14 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x6ACCD0 6472 10:43:14 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x6ADF10 6473 10:43:14 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x6ADF90 6474 10:43:14 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x6ADAA0 6475 10:43:14 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6476 10:43:14 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6ACCD0 6477 10:43:14 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6478 10:43:14 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6ADF10 6479 10:43:14 12/19/2012 IN TraceP_PL-3 "Activating 6480 10:43:14 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6ADF90 6481 10:43:14 12/19/2012 IN TraceP_PL-4 "Activating 6482 10:43:14 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6ADAA0 6483 10:43:14 12/19/2012 IN TraceP_SC-1 "Activating 6484 10:43:14 12/19/2012 IN TraceP_SC-2 "Activating 6485 10:43:14 12/19/2012 ER TraceP_PL-3 "Unable to add filter for session TestProf_1. error: 512 6486 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:59383:50886/./. 6487 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6488 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6489 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6490 10:43:14 12/19/2012 ER TraceP_PL-4 "Unable to add filter for session TestProf_1. error: 512 6491 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:59383:50886/./. 6492 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6493 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6494 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6495 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6496 10:43:14 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6497 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6498 10:43:14 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6499 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory PL-3 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6500 10:43:14 12/19/2012 ER TraceP_SC-1 "Unable to add filter for session TestProf_1. error: 512 6501 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:59383:50886/./. 6502 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6503 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6504 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6505 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6506 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6507 10:43:14 12/19/2012 ER TraceP_SC-2 "Unable to add filter for session TestProf_1. error: 512 6508 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:59383:50886/./. 6509 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6510 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6511 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6512 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6513 10:43:14 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6514 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6515 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-13774-20121219-104314 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6516 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6517 10:43:14 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6518 10:43:14 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6519 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6520 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-13774-20121219-104314 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6521 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory PL-4 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6522 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 6523 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-13841-20121219-104314 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6524 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13841-20121219-104314 has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6525 10:43:14 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestApp_Fork1-13841-20121219-104314 6526 10:43:14 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestApp_Fork1-13841-20121219-104314 is being deleted 6527 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13841-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6528 10:43:14 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6529 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory SC-2 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6530 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6531 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-16605-20121219-104314 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6532 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6533 10:43:14 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6534 10:43:14 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6535 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6536 10:43:14 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-16605-20121219-104314 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6537 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13783-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6538 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-13833-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6539 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-16606-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6540 10:43:14 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-13776-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6541 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6542 10:43:14 12/19/2012 IN TraceP_SC-1 "DBG: Return for invocation -3 value: 1, return code: 1 , SA_AIS_OK 6543 10:43:14 12/19/2012 IN TraceP_SC-1 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6544 10:43:14 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6545 10:43:14 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6546 10:43:15 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6547 10:43:15 12/19/2012 NO TraceC "Session TestProf_1 activated. 6548 10:43:15 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13775-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6549 10:43:15 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_type1-13831-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6550 10:43:15 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_type1-16604-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6551 10:43:15 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6552 10:43:15 12/19/2012 IN TraceP_PL-3 "DBG: Return for invocation -1 value: 1, return code: 1 , SA_AIS_OK 6553 10:43:15 12/19/2012 IN TraceP_PL-3 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6554 10:43:15 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13832-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6555 10:43:15 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6556 10:43:15 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6557 10:43:15 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6558 10:43:15 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6559 10:43:15 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-16613-20121219-104314 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6560 10:43:15 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6561 10:43:15 12/19/2012 IN TraceP_PL-4 "DBG: Return for invocation -2 value: 1, return code: 1 , SA_AIS_OK 6562 10:43:15 12/19/2012 IN TraceP_PL-4 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6563 10:43:15 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6564 10:43:15 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6565 10:43:15 12/19/2012 IN TraceP_SC-2 "DBG: Return for invocation -4 value: 1, return code: 1 , SA_AIS_OK 6566 10:43:15 12/19/2012 IN TraceP_SC-2 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6567 10:43:15 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6568 10:43:15 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6569 10:43:15 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6570 10:43:15 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 Wed Dec 19 10:43:15 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 1484 26613 0 10:43 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 1503 1484 0 10:43 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 1534 327 0 10:43 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:59383 -D tcp://192.168.0.1:50886 -o /storage/no-backup/tracecc-apr9010500/TestProf_1 root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:43:15 EST 2012 Allow session to run for 10 sec... DEBUG2: Consumer relayd socket sent [in consumer_send_relayd_socket() at consumer.c:708] DEBUG3: Relayd closing socket 21 [in relayd_close() at relayd.c:328] DEBUG3: Relayd connect ... [in relayd_connect() at relayd.c:315] DEBUG3: Creating relayd data socket from URI [in create_connect_relayd() at cmd.c:502] DEBUG3: Sending relayd sock info to consumer on 20 [in consumer_send_relayd_socket() at consumer.c:690] DEBUG1: Relayd session created with id 4 [in relayd_create_session() at relayd.c:144] DEBUG1: Consumer control socket created successfully with net idx 2 (fd: 18) [in consumer_add_relayd_socket() at consumer.c:2848] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Consumer adding relayd socket (idx: 2) [in consumer_add_relayd_socket() at consumer.c:2731] DEBUG3: Sending relayd socket file descriptor to consumer [in consumer_send_relayd_socket() at consumer.c:702] DEBUG2: Consumer relayd socket sent [in consumer_send_relayd_socket() at consumer.c:708] DEBUG3: Relayd closing socket 21 [in relayd_close() at relayd.c:328] DEBUG1: Starting all UST traces [in ust_app_start_trace_all() at ust-app.c:2518] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health state current 110, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 5, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] 6571 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory SC-1 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6572 10:43:16 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6573 10:43:16 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-892-20121219-104316 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6574 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6575 10:43:16 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6576 10:43:16 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6577 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6578 10:43:16 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-892-20121219-104316 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust DEBUG1: UST registration received with pid:892 ppid:1 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: UST registration received with pid:891 ppid:1 uid:0 gid:0 sock:21 name:TestApp_type1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: Futex n to 1 wait done [in futex_nto1_wait() at futex.c:90] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Dispatching UST registration pid:892 ppid:1 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Dispatching UST registration pid:891 ppid:1 uid:0 gid:0 sock:21 name:TestApp_type1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1310] DEBUG1: UST registration received with pid:900 ppid:892 uid:0 gid:0 sock:22 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: UST registration received with pid:893 ppid:1 uid:0 gid:0 sock:23 name:TestApp_Thread1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG3: Created hashtable size 4 at 0x650a50 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:892 ppid:1 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in ust_app_register() at ust-app.c:1494] DEBUG1: Futex n to 1 wait done [in futex_nto1_wait() at futex.c:90] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Dispatching UST registration pid:900 ppid:892 uid:0 gid:0 sock:22 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Dispatching UST registration pid:893 ppid:1 uid:0 gid:0 sock:23 name:TestApp_Thread1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1310] DEBUG2: UST app PID 892 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] DEBUG2: UST app global update for app sock 17 for session id 1 [in ust_app_global_update() at ust-app.c:2603] DEBUG2: UST app pid: 892 session id 1 not found, creating it [in create_ust_app_session() at ust-app.c:1048] DEBUG3: Created hashtable size 4 at 0x650f40 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x658290 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x6587d0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:892 and sock:17 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:892 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 892 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 892 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 892 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 8) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 9) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 26 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 26, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 26 stream 28 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (key 28, shm_fd 21, wait_fd 22, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 61 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (21,22) with relayd id 61 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 22 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG3: Adding metadata stream 22 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Metadata poll wait with 2 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 24 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 24, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 30 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 24 stream 30 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (key 30, shm_fd 21, wait_fd 23, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 62 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (21,23) with relayd id 62 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 30 [in consumer_add_stream() at consumer.c:616] DEBUG1: Consumer command ADD_STREAM chan 24 stream 32 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 2 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (key 32, shm_fd 21, wait_fd 24, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 63 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (21,24) with relayd id 63 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 32 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 3 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 24 stream 34 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (key 34, shm_fd 21, wait_fd 25, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 64 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (21,25) with relayd id 64 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 34 [in consumer_add_stream() at consumer.c:616] DEBUG1: Consumer command ADD_STREAM chan 24 stream 36 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 4 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (key 36, shm_fd 21, wait_fd 26, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEDEBUG2: UST app PID 891 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] DEBUG2: UST app globDEBUG1: Relayd stream added successfully with handle 65 [in relayd_add_stream() at relayd.DEBUG2:DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-104316/ (21,26) with relayd id DEBUG3: Created hashtable size 4 at 0x666300 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x666af0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x667030 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:891 and sock:21 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:891 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 891 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 891 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 891 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 8) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 9) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 40 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 40, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 40 stream 42 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-104316/ (key 42, shm_fd 21, wait_fd 27, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 66 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-104316/ (21,27) with relayd id 66 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 27 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG3: Adding metadata stream 27 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: Metadata poll wait with 3 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: consumer_add_channel 38 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 38, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_type1-891-20121219-104316/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 44 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 38 stream 44 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-104316/ (key 44, shm_fd 21, wait_fd 28, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 67 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-104316/ (21,28) with relayd id 67 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 44 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 6 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 46 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-104316/ (key 46, shm_fd 21, wait_fd 29, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 68 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-104316/ (21,29) with relayd id 68 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 46 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 7 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 48 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-104316/ (key 48, shm_fd 21, wait_fd 30, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 69 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-104316/ (21,30) with relayd id 69 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 48 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 8 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 50 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-104316/ (key 50, shm_fd 21, wait_fd 31, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 70 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-104316/ (21,31) with relayd id 70 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in coDEBUG1: Apps with sock 21 added to poll set [in thDEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 50 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 9 fd [in consumer_thread_data_poll() at consumer.c:2341] .c:987] DEBUG3: Created hashtable size 4 at 0x66f780 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x66fcc0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:900 and sock:22 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:900 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 900 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 900 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 900 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 8) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 9) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 54 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 54, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 54 stream 56 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (key 56, shm_fd 21, wait_fd 32, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 71 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (21,32) with relayd id 71 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 32 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG3: Adding metadata stream 32 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Metadata poll wait with 4 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 52 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 52, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 58 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 52 stream 58 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (key 58, shm_fd 21, wait_fd 33, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 72 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (21,33) with relayd id 72 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 58 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 10 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 52 stream 60 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (key 60, shm_fd 21, wait_fd 34, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 73 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (21,34) with relayd id 73 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 60 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 11 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 52 stream 62 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (key 62, shm_fd 21, wait_fd 35, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 74 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (21,35) with relayd id 74 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 62 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 12 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 52 stream 64 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (key 64, shm_fd 21, wait_fd 36, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 75 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-104316/ (21,36) with relayd id 75 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 64 [in consDEBUG1: App registered with pid:893 pDEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] ajor version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] tive FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: AcDEBUG3: Created hashtable size 4 at 0x677c70 of type 0 [in lttng_ht_new() at hashtable.c:96] mer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 13 fd [in consumer_thread_data_poll() at consumer.c:2341] T app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:1077] 6579 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_type1-891-20121219-104316 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6580 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-900-20121219-104316 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6581 10:43:16 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-893-20121219-104316 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust DEBUG2: UST app channel channel0 created successfully for pid:893 and sock:23 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:893 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 893 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 893 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 893 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 8) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 9) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 68 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 68, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 68 stream 70 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (key 70, shm_fd 21, wait_fd 37, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 76 [in relayd_add_stream() at relayd.c:199] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (21,37) with relayd id 76 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Adding metadata stream 37 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG3: Adding metadata stream 37 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: consumer_add_channel 66 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Metadata poll wait with 5 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Allocated channel (key 66, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 72 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 66 stream 72 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (key 72, shm_fd 21, wait_fd 38, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 77 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (21,38) with relayd id 77 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 72 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 14 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 66 stream 74 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (key 74, shm_fd 21, wait_fd 39, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 78 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (21,39) with relayd id 78 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 74 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Consumer command ADD_STREAM chan 66 stream 76 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG2: Sending stream 78 of channel channel0 to kernel consumer [in sendDEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 15 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (key 76, shm_fd 21, wait_fd 40, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 79 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (21,40) with relayd id 79 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 76 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 16 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 66 stream 78 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (key 78, shm_fd 21, wait_fd 41, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 80 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-104316/ (21,41) with relayd id 80 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 78 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 41 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 494, last 110, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 29 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 29, stream key: 46) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] DEBUG1: Consumer mmap write() ret 4042 (len 4042) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] 6582 10:43:24 12/19/2012 IN TraceC "DBG: request to rescan directory 6583 10:43:24 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory .tmp is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500 6584 10:43:24 12/19/2012 IN TraceC "DBG: Jobs took a long time to execute: 1384827us Wed Dec 19 10:43:25 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 1484 26613 0 10:43 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 1503 1484 0 10:43 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 1534 327 0 10:43 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:59383 -D tcp://192.168.0.1:50886 -o /storage/no-backup/tracecc-apr9010500/TestProf_1 root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:43:25 EST 2012 Printing Session #################### Wed Dec 19 10:43:25 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:43:14-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress : "" 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-104316 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-104316 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-104316 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-104316 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-104314 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-104314 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-104314 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 120K /storage/no-backup/tracecc-apr9010500/TestProf_1 Wed Dec 19 10:43:25 EST 2012 Printing Ports while Session is active #################### tcp 0 0 192.168.0.1:50886 0.0.0.0:* LISTEN 0 575623 1534/lttng-relayd tcp 0 0 192.168.0.1:59383 0.0.0.0:* LISTEN 0 575622 1534/lttng-relayd tcp 0 0 192.168.0.1:59383 192.168.0.4:60655 ESTABLISHED 0 575627 1534/lttng-relayd tcp 0 0 192.168.0.1:55800 192.168.0.1:59383 ESTABLISHED 0 574947 1503/lttng-consumer tcp 0 0 192.168.0.1:59383 192.168.0.2:47020 ESTABLISHED 0 575629 1534/lttng-relayd tcp 0 0 192.168.0.1:50886 192.168.0.1:60129 ESTABLISHED 0 575632 1534/lttng-relayd tcp 0 0 192.168.0.1:59383 192.168.0.3:33846 ESTABLISHED 0 575625 1534/lttng-relayd tcp 0 0 192.168.0.1:50886 192.168.0.2:46568 ESTABLISHED 0 575631 1534/lttng-relayd tcp 0 0 192.168.0.1:50886 192.168.0.3:38850 ESTABLISHED 0 575626 1534/lttng-relayd tcp 0 0 192.168.0.1:59383 192.168.0.1:55800 ESTABLISHED 0 575628 1534/lttng-relayd tcp 0 0 192.168.0.1:50886 192.168.0.4:45798 ESTABLISHED 0 575630 1534/lttng-relayd tcp 0 0 192.168.0.1:60129 192.168.0.1:50886 ESTABLISHED 0 576753 1503/lttng-consumer Wed Dec 19 10:43:25 EST 2012 Deactivating Profile #################### Wed Dec 19 10:43:25 EST 2012 Dec 19 10:43:25 SC-1 osafimmnd[6451]: Ccb 528 COMMITTED (TraceC) Dec 19 10:43:25 PL-3 osafimmnd[4905]: Ccb 528 COMMITTED (TraceC) Dec 19 10:43:25 PL-4 osafimmnd[4935]: Ccb 528 COMMITTED (TraceC) Dec 19 10:43:25 SC-2 osafimmnd[5121]: Ccb 528 COMMITTED (TraceC) Dec 19 10:43:25 SC-1 osafimmnd[6451]: Ccb 529 COMMITTED (TraceC) Dec 19 10:43:25 PL-3 osafimmnd[4905]: Ccb 529 COMMITTED (TraceC) Dec 19 10:43:25 PL-4 osafimmnd[4935]: Ccb 529 COMMITTED (TraceC) Dec 19 10:43:25 SC-2 osafimmnd[5121]: Ccb 529 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 502, last 110, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 13 [in process_client_msg() at main.c:2204] DEBUG1: Counting number of available session for UID 0 GID 0 [in lttng_sessions_count() at main.c:2175] DEBUG1: Getting all available session for UID 0 GID 0 [in cmd_list_lttng_sessions() at cmd.c:2074] DEBUG1: Sending response (size: 4388, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 17 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Stopping all UST traces [in ust_app_stop_trace_all() at ust-app.c:2544] DEBUG1: Stopping tracing for ust app pid 900 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 892 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 893 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 891 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 24 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Consumer data pending for id 1 [in consumer_is_data_pending() at consumer.c:784] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST consumer data pending command for id 1 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:372] DEBUG1: Consumer data pending command on session id 1 [in consumer_data_pending() at consumer.c:2956] DEBUG1: Relayd begin data pending [in relayd_begin_data_pending() at relayd.c:522] DEBUG3: Relayd sending command 10 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 62 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 62 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 63 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 63 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 64 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 64 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 65 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 65 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 67 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 67 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 68 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 68 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 69 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 69 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 70 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 70 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 72 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 72 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 73 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 73 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 74 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 74 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 75 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 75 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 77 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 77 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 78 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 78 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 79 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 79 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 80 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 80 [in relayd_data_pending() at relayd.c:459] DEBUG1: Relayd end data pending [in relayd_end_data_pending() at relayd.c:569] DEBUG3: Relayd sending command 11 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd end data pending is data inflight: 0 [in relayd_end_data_pending() at relayd.c:593] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Consumer data pending ret 0 [in consumer_is_data_pending() at consumer.c:820] DEBUG1: Sending response (size: 16, retcode: Unknown error code) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:700] DEBUG2: Sending destroy relayd command to consumer... [in consumer_send_destroy_relayd() at consumer.c:79] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST consumer destroying relayd 2 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:331] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG2: Consumer send destroy relayd command done [in consumer_send_destroy_relayd() at consumer.c:102] DEBUG1: Destroy all UST traces [in ust_app_destroy_trace_all() at ust-app.c:2570] DEBUG1: Destroy tracing for ust app pid 900 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 4 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG1: Destroy tracing for ust app pid 892 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 4 [in trace_ust_destroy_metadata() at trace-ust.c:586] nsumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: poll num_rdy : 4 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: fd 36 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: In read_subbuffer (wait_fd: 32, stream key: 56) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd sending metadata of size 2401 [in relayd_send_metadata() at relayd.c:287] DEBUG1: Polling fd 36 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 35 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 35 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 33 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG3: Relayd sending command 6 of size 24 [in send_command() at relayd.c:74] DEBUG1: Polling fd 33 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 34 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 34 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 4 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 36 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 36, stream key: 64) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] DEBUG1: Consumer mmap write() ret 68 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Normal read on fd 35 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 35, stream key: 62) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] DEBUG1: Consumer mmap write() ret 68 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Normal read on fd 33 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 33, stream key: 58) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] DEBUG1: Consumer mmap write() ret 68 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Normal read on fd 34 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 34, stream key: 60) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] DEBUG1: Consumer mmap write() ret 68 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Polling fd 36 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Polling fd 35 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Polling fd 33 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Polling fd 34 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 4 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 36 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 36, stream key: 64) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Normal read on fd 35 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 35, stream key: 62) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Normal read on fd 33 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 33, stream key: 58) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Normal read on fd 34 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 34, stream key: 60) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Polling fd 36 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 36 [in consumer_del_stream() at consumer.c:401] DEBUG1: Metadata stream id 71 with padding 1707 written before data [in write_relayd_metadata_id() at consumer.c:1325] DEBUG1: Consumer mmap write() ret 2389 (len 2389) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: In read_subbuffer (wait_fd: 32, stream key: 56) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG3: Consumer delete metadata stream 32 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Relayd closing stream id 75 [in relayd_send_close_stream() at relayd.c:381] DEBUG3: Relayd sending command 7 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Destroy tracing for ust app pid 893 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 4 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG1: Destroy tracing for ust app pid 891 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 4 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG2: Trace UST destroy session 1 [in trace_ust_destroy_session() at trace-ust.c:668] DEBUG1: Destroying session TestProf_1 [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] Error: Relayd close stream replied error -526032384 DEBUG1: Relayd close stream id 75 successfully [in relayd_send_close_stream() at relayd.c:409] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Polling fd 35 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 35 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 74 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Polling fd 33 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 33 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 72 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Polling fd 34 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 34 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 73 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 52 [in consumer_del_channel() at consumer.c:822] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 41 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() Dec 19 10:43:25 SC-1 osafimmnd[6451]: Ccb 530 COMMITTED (TraceC) at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 13 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 12 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: fd 30 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Relayd closing stream id 71 [in relayd_send_close_stream() at relayd.c:381] DEBUG1: Polling fd 30 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 41 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metDec 19 10:43:25 PL-3 osafimmnd[4905]: Ccb 530 COMMITTED (TraceC) adata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 54 [in consumer_del_channel() at consumer.c:822] DEBUG1: Polling fd 41 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 29 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 29 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 23 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Metadata poll wait with 4 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Polling fd 23 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Metadata fd 22 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: fd 24 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: In read_subbuffer (wait_fd: 22, stream key: 28) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Polling fd 24 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 25 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 25 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 38 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Relayd sending metadata of size 2401 [in relayd_send_metadata() at relayd.c:287] DEBUG1: Polling fd 38 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 31 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 31 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 39 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2389) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2389) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Polling fd 39 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG3: Consumer delete metadata stream 22 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Relayd closing stream id 61 [in relayd_send_close_stream() at relayd.c:381] DEBUG1: fd 28 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 26 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata fd 37 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: Polling fd 28 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 26 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: In read_subbuffer (wait_fd: 37, stream key: 70) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd sending metadata of size 2403 [in relayd_send_metadata() at relayd.c:287] DEBUG1: Polling fd 26 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 40 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2391) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Polling fd 40 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: polling on 13 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 12 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 30 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 30, stream key: 48) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2391) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG3: Consumer delete metadata stream 37 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Connection reset by peer [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 272) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 272) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 30 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 76 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 68 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata fd 27 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: In read_subbuffer (wait_fd: 27, stream key: 42) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd closing stream id 69 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 41 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: Relayd sending metadata of size 2848 [in relayd_send_metadata() at relayd.c:287] DEBUG1: In read_subbuffer (wait_fd: 41, stream key: 78) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1:Dec 19 10:43:25 PL-4 osafimmnd[4935]: Ccb 530 COMMITTED (TraceC) Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer mmap write() ret -1 (len 2836) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2836) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG3: Consumer delete metadata stream 27 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Consumer del stream 41 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 66 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 40 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata poll wait with 1 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Relayd closing stream id 80 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 29 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 29, stream key: 46) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 1693) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 1693) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 29 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 68 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 23 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 23, stream key: 30) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 23 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 62 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 24 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 24, stream key: 32) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 24 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 63 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 25 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 25, stream key: 34) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 25 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 64 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 38 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 38, stream key: 72) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 38 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 77 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 31 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 31, stream key: 50) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 337) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 337) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 31 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 70 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 39 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 39, stream key: 74) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 39 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 78 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 28 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 28, stream key: 44) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 28 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 67 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 38 [in consumer_del_channel() at consumer.c:822] DEBUG1: Normal read on fd 26 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 26, stream key: 36) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 26 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 65 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 24 [in consumer_del_channel() at consumer.c:822] DEBUG1: Normal read on fd 40 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 40, stream key: 76) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 40 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 79 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer destroy and close relayd socket pair [in destroy_relayd() at consumer.c:240] Dec 19 10:43:25 SC-2 osafimmnd[5121]: Ccb 530 COMMITTED (TraceC) DEBUG1: Consumer delete channel key 66 [in consumer_del_channel() at consumer.c:822] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: polling on 1 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG2: Trace destroy UST channel channel0 [in trace_ust_destroy_channel() at trace-ust.c:551] DEBUG3: Relayd closing socket 18 [in relayd_close() at relayd.c:328] DEBUG3: Relayd closing socket 20 [in relayd_close() at relayd.c:328] DEBUG2: Trace destroy UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA [in trace_ust_destroy_event() at trace-ust.c:510] 6585 10:43:25 12/19/2012 IN TraceC "DBG: Deactivating 6586 10:43:25 12/19/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProf_1,traceMId=1" from Registries 6587 10:43:25 12/19/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProf_1,traceMId=1" is valid 6588 10:43:25 12/19/2012 NO TraceC "Deactivating session TestProf_1. 6589 10:43:25 12/19/2012 IN TraceC "DBG: update Async for deactivate - first time 6590 10:43:25 12/19/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 6591 10:43:25 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x6B1E80 6592 10:43:25 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x6B1FB0 6593 10:43:25 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x6B20E0 6594 10:43:25 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x6B2210 6595 10:43:25 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6596 10:43:25 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6597 10:43:25 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B1E80 6598 10:43:25 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6599 10:43:25 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B1FB0 6600 10:43:25 12/19/2012 IN TraceP_PL-3 "Deactivating 6601 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: hasSession: Invoke lttng_list_session. 6602 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6603 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6604 10:43:25 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B20E0 6605 10:43:25 12/19/2012 IN TraceP_PL-4 "Deactivating 6606 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: hasSession: Invoke lttng_list_session. 6607 10:43:25 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B2210 6608 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6609 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6610 10:43:25 12/19/2012 IN TraceP_SC-1 "Deactivating 6611 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: hasSession: Invoke lttng_list_session. 6612 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6613 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6614 10:43:25 12/19/2012 IN TraceP_SC-2 "Deactivating 6615 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: hasSession: Invoke lttng_list_session. 6616 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6617 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6618 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6619 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6620 10:43:25 12/19/2012 IN TraceP_PL-3 "DBG: Return for invocation -5 value: 1, return code: 1 , SA_AIS_OK 6621 10:43:25 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6622 10:43:25 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6623 10:43:25 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6624 10:43:25 12/19/2012 IN TraceP_PL-3 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6625 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6626 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6627 10:43:25 12/19/2012 IN TraceP_PL-4 "DBG: Return for invocation -6 value: 1, return code: 1 , SA_AIS_OK 6628 10:43:25 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6629 10:43:25 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6630 10:43:25 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6631 10:43:25 12/19/2012 IN TraceP_PL-4 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6632 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6633 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6634 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6635 10:43:25 12/19/2012 IN TraceP_SC-2 "DBG: Return for invocation -8 value: 1, return code: 1 , SA_AIS_OK 6636 10:43:25 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6637 10:43:25 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6638 10:43:25 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6639 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6640 10:43:25 12/19/2012 IN TraceP_SC-1 "DBG: Return for invocation -7 value: 1, return code: 1 , SA_AIS_OK 6641 10:43:25 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6642 10:43:25 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6643 10:43:25 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6644 10:43:25 12/19/2012 IN TraceC "deactivateSessionOnTracePsComplete 6645 10:43:25 12/19/2012 IN TraceC "DBG: Terminating relayD for session TestProf_1 6646 10:43:25 12/19/2012 IN TraceP_SC-2 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6647 10:43:25 12/19/2012 IN TraceP_SC-1 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6648 10:43:25 12/19/2012 NO TraceC "Deactivate complete for session "TestProf_1" 6649 10:43:25 12/19/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProf_1,traceMId=1 6650 10:43:25 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6651 10:43:25 12/19/2012 IN TraceC "DBG: Received death of pid 1534 with ret 9. 6652 10:43:25 12/19/2012 IN TraceC "DBG: No handler for pid 1534. 6653 10:43:25 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Dec 19 10:43:26 EST 2012 Printing Profile #################### Wed Dec 19 10:43:26 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 12/19/12T10:43:25-0500 progress.timeActionStarted : 12/19/12T10:43:25-0500 progress.timeOfLastStatusUpdate : 12/19/12T10:43:25-0500 Wed Dec 19 10:43:26 EST 2012 Printing Session #################### Wed Dec 19 10:43:26 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:43:14-0500 state : COMPLETE stopTime : 12/19/12T10:43:25-0500 traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress : "" 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-104316 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-104316 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-104316 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-104316 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-104314 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-104314 20K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-104314 100K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 104K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-104314 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-104314 104K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 108K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-104314 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-104314 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-104314 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 276K /storage/no-backup/tracecc-apr9010500/TestProf_1 Wed Dec 19 10:43:26 EST 2012 Printing Ports after Session has been deactivated #################### Wed Dec 19 10:43:26 EST 2012 Converting Session #################### Wed Dec 19 10:43:26 EST 2012 6654 10:43:26 12/19/2012 IN TraceC "DBG: Converting output operation 6655 10:43:26 12/19/2012 IN TraceC "DBG: Try to async-remove directory "/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs" 6656 10:43:26 12/19/2012 IN TraceC "DBG: Directory /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs was created successfully! 6657 10:43:26 12/19/2012 NO TraceC "Converting session "TestProf_1" 6658 10:43:26 12/19/2012 IN TraceC "DBG: resetOrCreate traceSessionRecordingId=TestProf_1,traceMId=1 6659 10:43:26 12/19/2012 IN TraceC "DBG: ConvertAsyncProgress: Create request for child object of traceSessionRecordingId=TestProf_1,traceMId=1 6660 10:43:26 12/19/2012 IN TraceC "DBG: ConvertAsyncProgressImplementer: id=1,traceSessionRecordingId=TestProf_1,traceMId=1 was successfully created 6661 10:43:26 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6662 10:43:26 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/PL-3 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-3.log ==- 6663 10:43:26 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/PL-3,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-3.log === 6664 10:43:26 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory Converted_logs is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6665 10:43:26 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6666 10:43:26 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6667 10:43:26 12/19/2012 IN TraceC "DBG: Received death of pid 1629 with ret 0. 6668 10:43:26 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 0 6669 10:43:26 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 0 6670 10:43:26 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/PL-4 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-4.log ==- 6671 10:43:26 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/PL-4,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-4.log === DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 596, last 110, ret 0 [in health_check_state() at health.c:127] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] 6672 10:43:29 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6673 10:43:29 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6674 10:43:29 12/19/2012 IN TraceC "DBG: Received death of pid 1631 with ret 256. 6675 10:43:29 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6676 10:43:29 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6677 10:43:29 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6678 10:43:29 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/SC-1 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-1.log ==- 6679 10:43:29 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/SC-1,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-1.log === 6680 10:43:29 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6681 10:43:29 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6682 10:43:29 12/19/2012 IN TraceC "DBG: Received death of pid 1639 with ret 256. 6683 10:43:29 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6684 10:43:29 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6685 10:43:29 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6686 10:43:29 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/SC-2 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-2.log ==- 6687 10:43:29 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/SC-2,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-2.log === 6688 10:43:29 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6689 10:43:29 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6690 10:43:29 12/19/2012 IN TraceC "DBG: Received death of pid 1641 with ret 256. 6691 10:43:29 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6692 10:43:29 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6693 10:43:29 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6694 10:43:29 12/19/2012 NO TraceC "Completed conversion of session "TestProf_1". Wed Dec 19 10:43:30 EST 2012 Listing Session Directory Structure #################### /storage/no-backup/tracecc-apr9010500/TestProf_1: total 32 drwxr-xr-x 7 root root 4096 Dec 19 10:43 . drwxr-xr-x 5 root root 4096 Dec 19 10:43 .. drwxr-xr-x 2 root root 4096 Dec 19 10:43 Converted_logs drwxr-x--- 3 root root 4096 Dec 19 10:43 PL-3 drwxr-x--- 3 root root 4096 Dec 19 10:43 PL-4 drwxr-x--- 3 root root 4096 Dec 19 10:43 SC-1 drwxr-x--- 3 root root 4096 Dec 19 10:43 SC-2 -rw-r--r-- 1 root root 230 Dec 19 10:43 session_metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/Converted_logs: total 72 drwxr-xr-x 2 root root 4096 Dec 19 10:43 . drwxr-xr-x 7 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 26705 Dec 19 10:43 Node_PL-3.log -rwxr-xr-x 1 root root 26460 Dec 19 10:43 Node_PL-4.log -rwxr-xr-x 1 root root 0 Dec 19 10:43 Node_SC-1.log -rwxr-xr-x 1 root root 0 Dec 19 10:43 Node_SC-2.log -rw-r--r-- 1 root root 7555 Dec 19 10:43 conversionResult.log /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:43 . drwxr-xr-x 7 root root 4096 Dec 19 10:43 .. drwxr-x--- 6 root root 4096 Dec 19 10:43 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:43 . drwxr-x--- 3 root root 4096 Dec 19 10:43 .. drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-13775-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-13783-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Thread1-13776-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_type1-13774-20121219-104314 /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-104314: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-104314: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-104314: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-104314: total 32 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 8192 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:43 . drwxr-xr-x 7 root root 4096 Dec 19 10:43 .. drwxr-x--- 6 root root 4096 Dec 19 10:43 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:43 . drwxr-x--- 3 root root 4096 Dec 19 10:43 .. drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-13832-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-13841-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Thread1-13833-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_type1-13831-20121219-104314 /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-104314: total 24 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-104314: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-104314: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-104314: total 32 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 8192 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:43 . drwxr-xr-x 7 root root 4096 Dec 19 10:43 .. drwxr-x--- 6 root root 4096 Dec 19 10:43 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:43 . drwxr-x--- 3 root root 4096 Dec 19 10:43 .. drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-892-20121219-104316 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-900-20121219-104316 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Thread1-893-20121219-104316 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_type1-891-20121219-104316 /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-104316: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-104316: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-104316: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-104316: total 12 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:43 . drwxr-xr-x 7 root root 4096 Dec 19 10:43 .. drwxr-x--- 6 root root 4096 Dec 19 10:43 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:43 . drwxr-x--- 3 root root 4096 Dec 19 10:43 .. drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-16605-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Fork1-16613-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_Thread1-16606-20121219-104314 drwxr-x--- 2 root root 4096 Dec 19 10:43 TestApp_type1-16604-20121219-104314 /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-104314: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-104314: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-104314: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-104314: total 12 drwxr-x--- 2 root root 4096 Dec 19 10:43 . drwxr-x--- 6 root root 4096 Dec 19 10:43 .. -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:43 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:43 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:43 metadata Wed Dec 19 10:43:30 EST 2012 Printing Session #################### Wed Dec 19 10:43:30 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:43:14-0500 state : COMPLETE stopTime : 12/19/12T10:43:25-0500 traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress.actionName : convertOutput progress.additionalInfo : nodeName=All processId=All progress.info : Convert is terminated with failure! progress.percentage : 100 progress.result : FAILURE progress.resultInfo : Failed to convert output progress.state : FINISHED progress.timeActionCompleted : 12/19/12T10:43:29-0500 progress.timeActionStarted : 12/19/12T10:43:26-0500 progress.timeOfLastStatusUpdate : "" Wed Dec 19 10:43:30 EST 2012 Deleting Session #################### Wed Dec 19 10:43:30 EST 2012 Wed Dec 19 10:43:30 EST 2012 Deleting Profile #################### Wed Dec 19 10:43:30 EST 2012 Dec 19 10:43:30 SC-1 osafimmnd[6451]: Ccb 536 COMMITTED (TraceCLI) Dec 19 10:43:30 PL-3 osafimmnd[4905]: Ccb 536 COMMITTED (TraceCLI) Dec 19 10:43:30 PL-4 osafimmnd[4935]: Ccb 536 COMMITTED (TraceCLI) Dec 19 10:43:30 SC-2 osafimmnd[5121]: Ccb 536 COMMITTED (TraceCLI) ########################################################################### ########################################################################### ########################################################################### SC-1:/storage/no-backup/coremw/var/log/saflog # 6695 10:43:30 12/19/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProf_1,traceMId=1 6696 10:43:30 12/19/2012 IN TraceC "deleteSession 6697 10:43:30 12/19/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProf_1,traceMId=1 6698 10:43:30 12/19/2012 IN TraceC "DBG: Try to async-remove directory "/storage/no-backup/tracecc-apr9010500//TestProf_1" 6699 10:43:30 12/19/2012 IN TraceC "DBG: remove called to delete file /storage/no-backup/tracecc-apr9010500//.tmp/TestProf_1 6700 10:43:30 12/19/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProf_1,traceMId=1 6701 10:43:30 12/19/2012 IN TraceC "DBG: Try to remove directory "/storage/no-backup/tracecc-apr9010500//.tmp/TestProf_1" 6702 10:43:30 12/19/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProf_1,traceMId=1 6703 10:43:30 12/19/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProf_1,traceMId=1 6704 10:43:30 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6705 10:43:30 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProf_1 has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500 6706 10:43:30 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProf_1 6707 10:43:30 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProf_1 6708 10:43:30 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProf_1 is being deleted 6709 10:43:31 12/19/2012 IN TraceC "DBG: Remove Directory "/storage/no-backup/tracecc-apr9010500//.tmp/TestProf_1" Deleted! 6710 10:43:31 12/19/2012 IN TraceC "DBG: Completed deletion of "/storage/no-backup/tracecc-apr9010500//.tmp/TestProf_1" DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 606, last 110, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 614, last 110, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 622, last 614, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 630, last 614, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 636, last 614, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 646, last 614, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 654, last 646, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 662, last 646, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 670, last 646, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 678, last 646, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 686, last 646, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 694, last 686, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 700, last 686, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 710, last 686, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 716, last 686, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 726, last 686, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 734, last 726, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 742, last 726, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 750, last 726, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 758, last 726, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 764, last 758, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 772, last 758, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 782, last 758, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 790, last 758, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 796, last 790, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 806, last 790, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 814, last 790, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 822, last 790, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 828, last 822, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health state current 838, last 822, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 846, last 822, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 854, last 822, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 862, last 822, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 870, last 862, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 876, last 862, ret 0 [in health_check_state() at health.c:127] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 886, last 862, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 894, last 862, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 902, last 862, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 908, last 902, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 53, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 33, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # fg lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 ^CSC-1:/storage/no-backup/coremw/var/log/saflog # DEBUG1: ignoring first SIGINT [in sighandler() at lttng-consumerd.c:78] DEBUG1: SIGINT caught [in sighandler() at main.c:3781] DEBUG1: Terminating all threads [in stop_threads() at main.c:382] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Health check thread dying [in thread_manage_health() at main.c:3114] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Notifying applications of session daemon state: 0 [in notify_ust_apps() at main.c:496] DEBUG1: Client thread dying [in thread_manage_clients() at main.c:3377] DEBUG1: consumer thread cleanup completed [in thread_manage_consumer() at main.c:1092] DEBUG1: Got the wait shm fd 6 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 0 [in futex_wait_update() at futex.c:63] DEBUG1: Futex n to 1 wait done [in futex_nto1_wait() at futex.c:90] DEBUG1: Dispatch thread dying [in thread_dispatch_ust_registration() at main.c:1360] DEBUG1: Application communication apps thread cleanup complete [in thread_manage_apps() at main.c:1284] DEBUG1: UST Registration thread cleanup complete [in thread_registration_apps() at main.c:1561] DEBUG1: Cleaning up [in cleanup() at main.c:402] DEBUG1: Removing /var/run/lttng directory [in cleanup() at main.c:407] at consumer.c:2631] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG1: Consumer delete flagged data stream [in validate_endpoint_status_data_stream() at consumer.c:2024] DEBUG1: polling thread exiting [in consumer_thread_data_poll() at consumer.c:2500] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Metadata thread pipe hung up [in consumer_thread_metadata_poll() at consumer.c:2141] DEBUG1: Metadata poll thread exiting [in consumer_thread_metadata_poll() at consumer.c:2253] PERROR: sendmsg: Broken pipe [in lttcomm_send_unix_sock() at unix.c:214] DEBUG1: Consumer destroying it. Closing everything. [in lttng_consumer_destroy() at consumer.c:1268] DEBUG1: Cleaning up all sessions [in cleanup() at main.c:421] DEBUG1: Closing all UST sockets [in cleanup() at main.c:434] DEBUG2: UST app cleaning registered apps hash table [in ust_app_clean_list() at ust-app.c:1757] DEBUG1: *** assert failed :-) *** ==> Matthew, BEET driven development works! [in cleanup() at main.c:452] Dec 19 10:46:49 SC-1 logger: lttng-sessiond is started SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # jobs [1] Running tail -f saLogSystem_20121218_145953.log & [2] Running tail -f /var/log/SC-1/messages & [3] Running tail -f /var/log/SC-2/messages & [4] Running tail -f /var/log/PL-3/messages & [5]- Running tail -f /var/log/PL-4/messages & [6]+ Running tail -f /cluster/temp/eusgarc/sc1_sessiond.log & SC-1:/storage/no-backup/coremw/var/log/saflog # fg tail -f /cluster/temp/eusgarc/sc1_sessiond.log ^C SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # jobs [1] Running tail -f saLogSystem_20121218_145953.log & [2] Running tail -f /var/log/SC-1/messages & [3] Running tail -f /var/log/SC-2/messages & [4]- Running tail -f /var/log/PL-3/messages & [5]+ Running tail -f /var/log/PL-4/messages & SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # 6711 10:48:24 12/19/2012 IN TraceC "DBG: request to rescan directory 6712 10:48:24 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory .tmp is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500 6713 10:48:24 12/19/2012 IN TraceC "DBG: !!! Jobs took excessively long to complete: 48926868us !!! SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # rm /cluster/temp/eusgarc/sc1_sessiond.log SC-1:/storage/no-backup/coremw/var/log/saflog # touch /cluster/temp/eusgarc/sc1_sessiond.log SC-1:/storage/no-backup/coremw/var/log/saflog # tail -f /cluster/temp/eusgarc/sc1_sessiond.log & [6] 2113 SC-1:/storage/no-backup/coremw/var/log/saflog # jobs [1] Running tail -f saLogSystem_20121218_145953.log & [2] Running tail -f /var/log/SC-1/messages & [3] Running tail -f /var/log/SC-2/messages & [4] Running tail -f /var/log/PL-3/messages & [5]- Running tail -f /var/log/PL-4/messages & [6]+ Running tail -f /cluster/temp/eusgarc/sc1_sessiond.log & SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # SC-1:/storage/no-backup/coremw/var/log/saflog # clear SC-1:/storage/no-backup/coremw/var/log/saflog # pkill -9 lttng-sessiond; usleep 90000; lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 & [7] 2153 SC-1:/storage/no-backup/coremw/var/log/saflog # sleep 2;/cluster/temp/eusgarc/test_once DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:3692] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:3940] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:3942] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:3995] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:3996] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:3997] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4006] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4008] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4017] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4019] DEBUG3: Created hashtable size 4 at 0x64a0a0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64a5c0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:3734] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:3734] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:3734] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:3826] DEBUG3: Session daemon client socket 8 and application socket 9 created [in init_daemon_socket() at main.c:3596] Warning: No tracing group detected DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:270] DEBUG1: Command subsystem initialized [in cmd_init() at cmd.c:2423] DEBUG1: [thread] Manage health check started [in thread_manage_health() at main.c:2953] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3147] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1300] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1378] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:496] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1310] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1107] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1135] DEBUG1: Got the wait shm fd 17 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:63] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] Wed Dec 19 10:50:38 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 2153 26613 0 10:50 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:50:38 EST 2012 Creating Profile #################### Wed Dec 19 10:50:38 EST 2012 6714 10:50:38 12/19/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 6715 10:50:38 12/19/2012 IN TraceC "DBG: newvalue size=23 6716 10:50:38 12/19/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProf 6717 10:50:38 12/19/2012 IN TraceC "DBG: traceExpressionsValidator was called for traceExpression with size = 1. 6718 10:50:38 12/19/2012 IN TraceC "DBG: conditionValidator: About to create a temporary session _TraceCValidator_5JhS69 in path /tmp/_TraceCValidator_5JhS69 - Invoke lttng_create_session 6719 10:50:38 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_create_handle for temporary session _TraceCValidator_5JhS69. 6720 10:50:38 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_set_default_handle_attr for temporary session _TraceCValidator_5JhS69 Dec 19 10:50:38 SC-1 osafimmnd[6451]: Ccb 537 COMMITTED (TraceCLI) Dec 19 10:50:38 PL-3 osafimmnd[4905]: Ccb 537 COMMITTED (TraceCLI) Dec 19 10:50:38 PL-4 osafimmnd[4935]: Ccb 537 COMMITTED (TraceCLI) Dec 19 10:50:38 SC-2 osafimmnd[5121]: Ccb 537 COMMITTED (TraceCLI) Wed Dec 19 10:50:38 EST 2012 Wait 1 sec... Dec 19 10:50:38 SC-1 osafimmnd[6451]: Ccb 538 COMMITTED (TraceC) Dec 19 10:50:38 PL-3 osafimmnd[4905]: Ccb 538 COMMITTED (TraceC) Dec 19 10:50:38 PL-4 osafimmnd[4935]: Ccb 538 COMMITTED (TraceC) Dec 19 10:50:38 SC-2 osafimmnd[5121]: Ccb 538 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2204] DEBUG1: Waiting for 1 URIs from client ... [in process_client_msg() at main.c:2711] DEBUG2: Trying to find session by name _TraceCValidator_5JhS69 [in session_find_by_name() at session.c:122] DEBUG1: Tracing session _TraceCValidator_5JhS69 created in (null) with ID 0 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name _TraceCValidator_5JhS69 [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x7f9cf000adb0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting trace directory path from URI to /tmp/_TraceCValidator_5JhS69 [in add_uri_to_consumer() at cmd.c:407] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2204] DEBUG1: Getting session _TraceCValidator_5JhS69 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name _TraceCValidator_5JhS69 [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2091] DEBUG3: Created hashtable size 4 at 0x7f9cf0003c00 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf0004140 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf0004680 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf0010350 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:239] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2043] DEBUG3: Created hashtable size 4 at 0x7f9cf0010350 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir /ust [in copy_session_consumer() at main.c:2061] DEBUG1: Spawning consumerd [in spawn_consumerd() at main.c:1722] DEBUG2: Consumer pid 2170 [in start_consumerd() at main.c:1903] DEBUG2: Spawning consumer control thread [in start_consumerd() at main.c:1906] DEBUG1: [thread] Manage consumer started [in thread_manage_consumer() at main.c:856] DEBUG1: Using 64-bit UST consumer at: /usr/lib64/lttng/libexec/lttng-consumerd [in spawn_consumerd() at main.c:1796] DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:270] DEBUG3: Created hashtable size 4 at 0x627030 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627570 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627ab0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x627ff0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Connecting to error socket /var/run/lttng/ustconsumerd64/error [in main() at lttng-consumerd.c:358] DEBUG3: Created hashtable size 4 at 0x628830 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Thread metadata poll started [in consumer_thread_metadata_poll() at consumer.c:2092] DEBUG1: Creating command socket /var/run/lttng/ustconsumerd64/command [in consumer_thread_sessiond_poll() at consumer.c:2545] DEBUG3: Created hashtable size 4 at 0x628d70 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Sending ready command to lttng-sessiond [in consumer_thread_sessiond_poll() at consumer.c:2558] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: polling on 1 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Metadata main loop started [in consumer_thread_metadata_poll() at consumerDEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:1DEBUG1: Connection on client_socket [in consumer_thread_sessiond_poll() at consumer.c:2581] nnel() at cmd.c:823] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:141] DEBUG3: Created hashtable size 4 at 0x7f9cf0011cc0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf0012200 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:303] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:277] DEBUG2: UST app adding channel channel0 to global domain for session id 0 [in ust_app_create_channel_glb() at ust-app.c:2043] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:302] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2204] DEBUG1: Getting session _TraceCValidator_5JhS69 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name _TraceCValidator_5JhS69 [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:700] DEBUG1: Destroy all UST traces [in ust_app_destroy_trace_all() at ust-app.c:2570] DEBUG2: Trace UST destroy session 0 [in trace_ust_destroy_session() at trace-ust.c:668] DEBUG1: Destroying session _TraceCValidator_5JhS69 [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG2: Trace destroy UST channel channel0 [in trace_ust_destroy_channel() at trace-ust.c:551] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] 6721 10:50:38 12/19/2012 IN TraceC "DBG: conditionValidator: Invoke lttng_destroy_session for temporary session _TraceCValidator_5JhS69 6722 10:50:38 12/19/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_5JhS69" 6723 10:50:38 12/19/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_5JhS69" Deleted! 6724 10:50:38 12/19/2012 IN TraceC "DBG: conditionValidator: Reply recieved from lttng_destroy_session for temporary session _TraceCValidator_5JhS69. 6725 10:50:38 12/19/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProf,traceMId=1 Wed Dec 19 10:50:39 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 2153 26613 0 10:50 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 2170 2153 0 10:50 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:50:39 EST 2012 Activating Profile #################### Wed Dec 19 10:50:39 EST 2012 Dec 19 10:50:39 SC-1 osafimmnd[6451]: Ccb 540 COMMITTED (TraceC) Dec 19 10:50:39 PL-3 osafimmnd[4905]: Ccb 540 COMMITTED (TraceC) Dec 19 10:50:39 SC-2 osafimmnd[5121]: Ccb 540 COMMITTED (TraceC) Dec 19 10:50:39 PL-4 osafimmnd[4935]: Ccb 540 COMMITTED (TraceC) Dec 19 10:50:39 SC-1 osafimmnd[6451]: Ccb 541 COMMITTED (TraceC) Dec 19 10:50:39 PL-3 osafimmnd[4905]: Ccb 541 COMMITTED (TraceC) Dec 19 10:50:39 PL-4 osafimmnd[4935]: Ccb 541 COMMITTED (TraceC) Dec 19 10:50:39 SC-2 osafimmnd[5121]: Ccb 541 COMMITTED (TraceC) Dec 19 10:50:39 SC-1 osafimmnd[6451]: Ccb 542 COMMITTED (TraceC) Dec 19 10:50:39 PL-3 osafimmnd[4905]: Ccb 542 COMMITTED (TraceC) Dec 19 10:50:39 PL-4 osafimmnd[4935]: Ccb 542 COMMITTED (TraceC) Dec 19 10:50:39 SC-2 osafimmnd[5121]: Ccb 542 COMMITTED (TraceC) Dec 19 10:50:40 SC-1 osafimmnd[6451]: Ccb 543 COMMITTED (TraceC) Dec 19 10:50:40 PL-3 osafimmnd[4905]: Ccb 543 COMMITTED (TraceC) Dec 19 10:50:40 PL-4 osafimmnd[4935]: Ccb 543 COMMITTED (TraceC) Dec 19 10:50:40 SC-2 osafimmnd[5121]: Ccb 543 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2204] DEBUG1: Waiting for 2 URIs from client ... [in process_client_msg() at main.c:2711] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Tracing session TestProf_1 created in (null) with ID 1 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x7f9cf0011b80 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer control URI set with port 52777 [in consumer_set_network_uri() at consumer.c:420] DEBUG3: Consumer set network uri subdir path SC-1/./ [in consumer_set_network_uri() at consumer.c:472] DEBUG3: Append domain trace name to subdir SC-1/./ [in add_uri_to_consumer() at cmd.c:402] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer data URI set with port 56915 [in consumer_set_network_uri() at consumer.c:429] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2091] DEBUG3: Created hashtable size 4 at 0x7f9cf00120c0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf000eeb0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf000f3f0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf000f910 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:239] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2043] DEBUG3: Created hashtable size 4 at 0x7f9cf000f910 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir SC-1/.//ust [in copy_session_consumer() at main.c:2061] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:179] DEBUG1: Enabling channel channel0 for session TestProf_1 [in cmd_enable_channel() at cmd.c:823] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:141] DEBUG3: Created hashtable size 4 at 0x7f9cf000fe80 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x7f9cf00103c0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:303] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:277] DEBUG2: UST app adding channel channel0 to global domain for session id 1 [in ust_app_create_channel_glb() at ust-app.c:2043] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:302] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 6 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:179] DEBUG2: Trace UST channel channel0 found by name [in trace_ust_find_channel_by_name() at trace-ust.c:136] DEBUG2: Trace UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA NOT found [in trace_ust_find_event() at trace-ust.c:174] DEBUG2: Trace UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, loglevel (1,10) created [in trace_ust_create_event() at trace-ust.c:379] DEBUG1: UST app creating event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA for all apps for session id 1 [in ust_app_create_event_glb() at ust-app.c:2171] DEBUG1: Event UST com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created in channel channel0 [in event_ust_enable_tracepoint() at event.c:476] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 16 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG2: Setting relayd for session TestProf_1 [in setup_relayd() at cmd.c:644] DEBUG3: Relayd connect ... [in relayd_connect() at relayd.c:315] DEBUG3: Creating relayd stream socket from URI [in create_connect_relayd() at cmd.c:492] DEBUG1: Relayd version check for major.minor 2.1 [in relayd_version_check() at relayd.c:219] DEBUG3: Relayd sending command 5 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 8 [in recv_reply() at relayd.c:90] DEBUG2: Relayd version is compatible [in relayd_version_check() at relayd.c:254] DEBUG3: Sending relayd sock info to consumer on 20 [in consumer_send_relayd_socket() at consumer.c:690] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Consumer adding relayd socket (idx: 2) [in consumer_add_relayd_socket() at consumer.c:2731] DEBUG3: Sending relayd socket file descriptor to consumer [in consumer_send_relayd_socket() at consumer.c:702] DEBUG2: Consumer relayd socket sent [in consumer_send_relayd_socket() at consumer.c:708] DEBUG3: Relayd closing socket 21 [in relayd_close() at relayd.c:328] DEBUG3: Relayd connect ... [in relayd_connect() at relayd.c:315] DEBUG3: Creating relayd data socket from URI [in create_connect_relayd() at cmd.c:502] DEBUG3: Sending relayd sock info to consumer on 20 [in consumer_send_relayd_socket() at consumer.c:690] DEBUG1: Relayd session created with id 4 [in relayd_create_session() at relayd.c:144] DEBUG1: Consumer control socket created successfully with net idx 2 (fd: 18) [in consumer_add_relayd_socket() at consumer.c:2848] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Consumer adding relayd socket (idx: 2) [in consumer_add_relayd_socket() at consumer.c:2731] DEBUG3: Sending relayd socket file descriptor to consumer [in consumer_send_relayd_socket() at consumer.c:702] DEBUG2: Consumer relayd socket sent [in consumer_send_relayd_socket() at consumer.c:708] DEBUG3: Relayd closing socket 21 [in relayd_close() at relayd.c:328] DEBUG1: Starting all UST traces [in ust_app_start_trace_all() at ust-app.c:2518] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Health state current 100, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 5, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] 6726 10:50:39 12/19/2012 IN TraceC "DBG: Activating 6727 10:50:39 12/19/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProf_1,traceMId=1 as a runtime object 6728 10:50:39 12/19/2012 NO TraceC "Activating Session TestProf_1. 6729 10:50:39 12/19/2012 IN TraceC "DBG: Directory /storage/no-backup/tracecc-apr9010500//TestProf_1 was created successfully! 6730 10:50:39 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6731 10:50:39 12/19/2012 IN TraceC "DBG: performActivate: Starting RelayD for session TestProf_1. 6732 10:50:39 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6733 10:50:39 12/19/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 52777 6734 10:50:39 12/19/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 56915 6735 10:50:39 12/19/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 6736 10:50:39 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6737 10:50:39 12/19/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProf_1" pid: 2189 bound on 192.168.0.1:52777:56915 6738 10:50:39 12/19/2012 IN TraceC "DBG: performActivate: Writing session into file 6739 10:50:39 12/19/2012 IN TraceC "DBG: Done Activating 6740 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProf_1 is being added to parent directory /storage/no-backup/tracecc-apr9010500 6741 10:50:39 12/19/2012 IN TraceC "DBG: RelayD has been launch for TestProf_1 session after 1 tries to connect to free ports. 6742 10:50:39 12/19/2012 IN TraceC "DBG: relayD ip address is found, IP address: 192.168.0.1 for host: SC-1 6743 10:50:39 12/19/2012 IN TraceC "DBG: performActivateAfterRelayD: Sending commands to the trace_ps. 6744 10:50:39 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x7FD278000980 6745 10:50:39 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x7FD278008130 6746 10:50:39 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x7FD2780081F0 6747 10:50:39 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x7FD278006A60 6748 10:50:39 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6749 10:50:39 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6750 10:50:39 12/19/2012 IN TraceC "DBG: Invoking admin op 0x7FD278000980 6751 10:50:39 12/19/2012 IN TraceC "DBG: Invoking admin op 0x7FD278008130 6752 10:50:39 12/19/2012 IN TraceP_PL-3 "Activating 6753 10:50:39 12/19/2012 IN TraceC "DBG: Invoking admin op 0x7FD2780081F0 6754 10:50:39 12/19/2012 IN TraceP_PL-4 "Activating 6755 10:50:39 12/19/2012 IN TraceC "DBG: Invoking admin op 0x7FD278006A60 6756 10:50:39 12/19/2012 IN TraceP_SC-1 "Activating 6757 10:50:39 12/19/2012 IN TraceP_SC-2 "Activating 6758 10:50:39 12/19/2012 ER TraceP_PL-3 "Unable to add filter for session TestProf_1. error: 512 6759 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:52777:56915/./. 6760 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6761 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6762 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6763 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6764 10:50:39 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6765 10:50:39 12/19/2012 ER TraceP_PL-4 "Unable to add filter for session TestProf_1. error: 512 6766 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:52777:56915/./. 6767 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6768 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6769 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6770 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory PL-3 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6771 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6772 10:50:39 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6773 10:50:39 12/19/2012 ER TraceP_SC-2 "Unable to add filter for session TestProf_1. error: 512 6774 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:52777:56915/./. 6775 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6776 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6777 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6778 10:50:39 12/19/2012 ER TraceP_SC-1 "Unable to add filter for session TestProf_1. error: 512 6779 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: invoke lttng_create_session for session TestProf_1 with url net://192.168.0.1:52777:56915/./. 6780 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_create_handle for session TestProf_1 6781 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_set_default_attr for session TestProf_1. 6782 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_enable_channel for session TestProf_1 6783 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6784 10:50:39 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6785 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_enable_event_with_filter for session TestProf_1, event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA, with cond '' 6786 10:50:39 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Invoking lttng_start_tracing, session name TestProf_1 6787 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6788 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-13774-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6789 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6790 10:50:39 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6791 10:50:39 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6792 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 6793 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-13774-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6794 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory SC-2 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6795 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6796 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-16605-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6797 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6798 10:50:39 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6799 10:50:39 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6800 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 6801 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-16605-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6802 10:50:39 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6803 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory PL-4 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6804 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 6805 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-13841-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6806 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 6807 10:50:39 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6808 10:50:39 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6809 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 6810 10:50:39 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_Fork1-13841-20121219-105039 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6811 10:50:39 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13783-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6812 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-13776-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6813 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-16606-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6814 10:50:40 12/19/2012 IN TraceP_SC-1 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6815 10:50:40 12/19/2012 IN TraceP_SC-1 "DBG: Return for invocation -11 value: 1, return code: 1 , SA_AIS_OK 6816 10:50:40 12/19/2012 IN TraceP_SC-1 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6817 10:50:40 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6818 10:50:40 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6819 10:50:40 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6820 10:50:40 12/19/2012 NO TraceC "Session TestProf_1 activated. 6821 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-13833-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6822 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13775-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 6823 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_type1-16604-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6824 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_type1-13831-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6825 10:50:40 12/19/2012 IN TraceP_PL-3 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6826 10:50:40 12/19/2012 IN TraceP_PL-3 "DBG: Return for invocation -9 value: 1, return code: 1 , SA_AIS_OK 6827 10:50:40 12/19/2012 IN TraceP_PL-3 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6828 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-16613-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 6829 10:50:40 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6830 10:50:40 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6831 10:50:40 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-13832-20121219-105039 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 6832 10:50:40 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6833 10:50:40 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6834 10:50:40 12/19/2012 IN TraceP_SC-2 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6835 10:50:40 12/19/2012 IN TraceP_SC-2 "DBG: Return for invocation -12 value: 1, return code: 1 , SA_AIS_OK 6836 10:50:40 12/19/2012 IN TraceP_SC-2 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6837 10:50:40 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6838 10:50:40 12/19/2012 IN TraceP_PL-4 "DBG: activateSession: Reply recieved for lttng_start_tracing for session TestProf_1. 6839 10:50:40 12/19/2012 IN TraceP_PL-4 "DBG: Return for invocation -10 value: 1, return code: 1 , SA_AIS_OK 6840 10:50:40 12/19/2012 IN TraceP_PL-4 "Activation completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1 6841 10:50:40 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6842 10:50:40 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 6843 10:50:40 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6844 10:50:40 12/19/2012 IN TraceC "receiveTracePActivateResult result: 1 op_result 1 Wed Dec 19 10:50:40 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 2153 26613 0 10:50 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 2170 2153 0 10:50 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 2189 327 0 10:50 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:52777 -D tcp://192.168.0.1:56915 -o /storage/no-backup/tracecc-apr9010500/TestProf_1 root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:50:40 EST 2012 Allow session to run for 10 sec... DEBUG1: UST registration received with pid:891 ppid:1 uid:0 gid:0 sock:17 name:TestApp_type1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: UST registration received with pid:892 ppid:1 uid:0 gid:0 sock:21 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: UST registration received with pid:893 ppid:1 uid:0 gid:0 sock:22 name:TestApp_Thread1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG1: UST registration received with pid:900 ppid:892 uid:0 gid:0 sock:23 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1513] DEBUG1: Futex n to 1 wait done [in futex_nto1_wait() at futex.c:90] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Dispatching UST registration pid:891 ppid:1 uid:0 gid:0 sock:17 name:TestApp_type1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Dispatching UST registration pid:892 ppid:1 uid:0 gid:0 sock:21 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Dispatching UST registration pid:893 ppid:1 uid:0 gid:0 sock:22 name:TestApp_Thread1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Dispatching UST registration pid:900 ppid:892 uid:0 gid:0 sock:23 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1322] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1310] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1413] DEBUG3: Created hashtable size 4 at 0x64b810 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:891 ppid:1 uid:0 gid:0 sock:17 name:TestApp_type1 (version 2.0) [in ust_app_register() at ust-app.c:1494] DEBUG2: UST app PID 891 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] DEBUG2: UST app global update for app sock 17 for session id 1 [in ust_app_global_update() at ust-app.c:2603] DEBUG2: UST app pid: 891 session id 1 not found, creating it [in create_ust_app_session() at ust-app.c:1048] DEBUG3: Created hashtable size 4 at 0x64cda0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x64e000 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64e4b0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 2 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:891 and sock:17 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:891 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 891 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 891 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 891 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 4) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 5) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 26 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 26, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 26 stream 28 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-105041/ (key 28, shm_fd 21, wait_fd 22, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 61 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-105041/ (21,22) with relayd id 61 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 22 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG3: Adding metadata stream 22 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: consumer_add_channel 24 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Metadata poll wait with 2 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Allocated channel (key 24, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_type1-891-20121219-105041/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 30 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 24 stream 30 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-105041/ (key 30, shm_fd 21, wait_fd 23, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 62 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-105041/ (21,23) with relayd id 62 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 30 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 2 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 24 stream 32 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-105041/ (key 32, shm_fd 21, wait_fd 24, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 63 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-105041/ (21,24) with relayd id 63 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 32 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 3 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 24 stream 34 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-105041/ (key 34, shm_fd 21, wait_fd 25, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG2: Sending stream 36 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] 6845 10:50:41 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory SC-1 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6846 10:50:41 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory ust is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6847 10:50:41 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-891-20121219-105041 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6848 10:50:41 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust has been deleted, moved, replaced or a delayed creation notification has been received in /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6849 10:50:41 12/19/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting ust 6850 10:50:41 12/19/2012 IN TraceC "DBG: directoryDeleteHandler : directory ust is being deleted 6851 10:50:41 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory ust is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 6852 10:50:41 12/19/2012 IN TraceC "DBG: synchronizeDirectory : directory TestApp_type1-891-20121219-105041 is being added by sync to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust mer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-105041/ (21,25) with relayd id 64 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 34 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 4 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 24 stream 36 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_type1-891-20121219-105041/ (key 36, shm_fd 21, wait_fd 26, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 65 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_type1-891-20121219-105041/ (21,26) with relayd id 65 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 36 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_pollDEBUG2: UST app pid: 892 session id 1 not found, creating it [in create_ust_app_session() at ust-app.c:1048] DEBUG3: Created hashtable size 4 at 0x656310 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x656b00 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x657040 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 2 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:892 and sock:21 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:892 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 892 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 892 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 892 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 4) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 5) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 40 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 40, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 40 stream 42 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (key 42, shm_fd 21, wait_fd 27, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 66 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (21,27) with relayd id 66 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: Adding metadata stream 27 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG3: Adding metadata stream 27 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: Metadata poll wait with 3 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: consumer_add_channel 38 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 38, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 44 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 38 stream 44 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (key 44, shm_fd 21, wait_fd 28, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 67 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (21,28) with relayd id 67 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 44 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 6 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 46 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (key 46, shm_fd 21, wait_fd 29, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 68 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (21,29) with relayd id 68 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 46 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 7 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 48 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (key 48, shm_fd 21, wait_fd 30, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 69 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (21,30) with relayd id 69 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 48 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 8 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 38 stream 50 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (key 50, shm_fd 21, wait_fd 31, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 70 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-892-20121219-105041/ (21,31) with relayd id 70 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in coDEBUG1: Apps with sock 21 added to poll set [in thread_manage_apps() at main.c:1236] DEBUG1: Apps thread polling on 4 fds [in thread_manage_apps() at main.c:1135] DEBUG3: Created hashtable size 4 at 0x65d9f0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:893 ppid:1 uid:0 gid:0 sock:22 name:TestApp_Thread1 (version 2.0) [in ust_app_register() at ust-app.c:1494] DEBUG2: UST app PID 893 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] DEBUG2: UST app global update for app sock 22 for session id 1 [in ust_app_global_update() at ust-app.c:2603] DEBUG2: UST app pid: 893 session id 1 not found, creating it [in create_ust_app_session() at ust-app.c:1048] DEBUG3: Created hashtable size 4 at 0x65efa0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x65f790 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65fcd0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:373] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:900] DEBUG2: UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA not found on shadow copy channel [in shadow_copy_channel() at ust-app.c:925] DEBUG3: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA allocated [in alloc_ust_app_event() at ust-app.c:407] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 2 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:893 and sock:22 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:893 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 893 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 893 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 893 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 4) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 5) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 54 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 54, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 54 stream 56 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (key 56, shm_fd 21, wait_fd 32, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 71 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (21,32) with relayd id 71 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 32 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG3: Adding metadata stream 32 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: Metadata poll wait with 4 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: consumer_add_channel 52 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 52, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 58 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 52 stream 58 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (key 58, shm_fd 21, wait_fd 33, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 72 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (21,33) with relayd id 72 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 58 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 10 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 52 stream 60 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (key 60, shm_fd 21, wait_fd 34, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 73 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (21,34) with relayd id 73 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 60 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG2: Sending stream 64 of channel channel0 to kernel consumer [in sendDEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Consumer command ADD_STREAM chan 52 stream 62 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 11 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (key 62, shm_fd 21, wait_fd 35, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 74 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (21,35) with relayd id 74 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 62 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_arDEBUG2: UST trace started for app pid 893 [in ust_app_global_update() at ust-app.c:2669] DEBUG1: Apps with sock 22 added to poll set [in thread_manage_apps() at main.c:1236] DEBUG1: Apps thread polling on 5 fds [in thread_manage_apps() at main.c:1135] DEBUG3: Created hashtable size 4 at 0x6666f0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:900 ppid:892 uid:0 gid:0 sock:23 name:TestApp_Fork1 (version 2.0) [in ust_app_register() at ust-app.c:1494] DEBUG2: UST app PID 900 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2877] DEBUG2: UST app global update for app sock 23 for session id 1 [in ust_app_global_update() at ust-app.c:2603] DEBUG2: UST app pid: 900 session id 1 not found, creating it [in create_ust_app_session() at ust-app.c:1048] DEBUG3: Created hashtable size 4 at 0x667c80 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:958] DEBUG2: Channel channel0 not found on shadow session copy, creating it [in shadow_copy_session() at ust-app.c:987] DEBUG3: Created hashtable size 4 at 0x668450 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 75 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Thread1-893-20121219-105041/ (21,36) with relayd id 75 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_dataDEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:935] DEBUG2: UST app session created successfully with handle 2 [in create_ust_app_session() at ust-app.c:1077] DEBUG2: UST app channel channel0 created successfully for pid:900 and sock:23 [in create_ust_channel() at ust-app.c:786] DEBUG2: UST app event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA created successfully for pid:900 [in create_ust_event() at ust-app.c:826] DEBUG1: Starting tracing for ust app pid 900 [in ust_app_start_trace() at ust-app.c:2227] DEBUG2: UST metadata opened for app pid 900 [in create_ust_app_metadata() at ust-app.c:1350] DEBUG2: UST metadata stream object created for app pid 900 [in create_ust_app_metadata() at ust-app.c:1368] DEBUG2: UST stream 1 ready (handle: 7) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 2 ready (handle: 6) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 3 ready (handle: 4) [in ust_app_start_trace() at ust-app.c:2315] DEBUG2: UST stream 4 ready (handle: 5) [in ust_app_start_trace() at ust-app.c:2315] DEBUG1: Sending metadata stream fd to consumer on 20 [in ust_consumer_send_session() at ust-consumer.c:330] DEBUG1: UST consumer sending metadata stream fd [in send_metadata() at ust-consumer.c:223] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: consumer_add_channel 68 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 68, shm_fd -1, wait_fd -1, mmap_len 768, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 68 stream 70 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (key 70, shm_fd 21, wait_fd 37, mmap_len 13056, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name metadata [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Sending streams of channel channel0 to UST consumer [in send_channel_streams() at ust-consumer.c:147] DEBUG2: Sending channel channel0 to UST consumer [in send_channel() at ust-consumer.c:50] DEBUG1: Relayd stream added successfully with handle 76 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (21,37) with relayd id 76 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding channel [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:144] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Adding metadata stream 37 to poll set [in consumer_thread_metadata_poll() at consumer.c:2175] DEBUG3: Adding metadata stream 37 to hash table [in consumer_add_metadata_stream() at consumer.c:1960] DEBUG1: Metadata poll wait with 5 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: consumer_add_channel 66 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:176] DEBUG1: Allocated channel (key 66, shm_fd -1, wait_fd -1, mmap_len 816, max_sb_size 4096) [in consumer_allocate_channel() at consumer.c:914] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG3: UST network consumer subdir path: SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ [in send_channel_streams() at ust-consumer.c:173] DEBUG2: Sending stream 72 of channel channel0 to kernel consumer [in send_channel_stream() at ust-consumer.c:100] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: Consumer command ADD_STREAM chan 66 stream 72 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (key 72, shm_fd 21, wait_fd 38, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_0 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 77 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (21,38) with relayd id 77 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 72 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 14 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 66 stream 74 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (key 74, shm_fd 21, wait_fd 39, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_1 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 78 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (21,39) with relayd id 78 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 74 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 15 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 66 stream 76 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (key 76, shm_fd 21, wait_fd 40, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_2 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 79 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (21,40) with relayd id 79 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST Consumer adding stream [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:207] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 76 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 16 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: Consumer command ADD_STREAM chan 66 stream 78 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:240] DEBUG3: Allocated stream SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (key 78, shm_fd 21, wait_fd 41, mmap_len 21888, out_fd -1, net_seq_idx 2, session_id 1 [in consumer_allocate_stream() at consumer.c:592] DEBUG1: Relayd adding stream for channel name channel0_3 [in relayd_add_stream() at relayd.c:167] DEBUG3: Relayd sending command 1 of size 4384 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 12 [in recv_reply() at relayd.c:90] DEBUG1: Relayd stream added successfully with handle 80 [in relayd_add_stream() at relayd.c:199] DEBUG1: UST consumer ADD_STREAM SC-1/.//ust/TestApp_Fork1-900-20121219-105041/ (21,41) with relayd id 80 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:323] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2367] DEBUG3: Adding consumer stream 78 [in consumer_add_stream() at consumer.c:616] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 28 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 35 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 33 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 31 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 36 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 30 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 34 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 41 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 29 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] 6853 10:50:41 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-892-20121219-105041 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6854 10:50:41 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Thread1-893-20121219-105041 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 6855 10:50:42 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestApp_Fork1-900-20121219-105041 is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 494, last 100, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 502, last 100, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] Wed Dec 19 10:50:50 EST 2012 root 316 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_p root 327 1 0 10:38 ? 00:00:00 /opt/trace/sbin/trace_c root 2153 26613 0 10:50 pts/0 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 2170 2153 0 10:50 pts/0 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error root 2189 327 0 10:50 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:52777 -D tcp://192.168.0.1:56915 -o /storage/no-backup/tracecc-apr9010500/TestProf_1 root 21870 1 0 Dec18 ? 00:00:10 /opt/trace_ea/bin/TraceEa root 21878 21870 0 Dec18 ? 00:00:01 /opt/trace_ea/bin/TraceEa Wed Dec 19 10:50:50 EST 2012 Printing Session #################### Wed Dec 19 10:50:50 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:50:39-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress : "" 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-105041 20K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-105039 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-105039 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-105039 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 116K /storage/no-backup/tracecc-apr9010500/TestProf_1 Wed Dec 19 10:50:50 EST 2012 Printing Ports while Session is active #################### tcp 0 0 192.168.0.1:52777 0.0.0.0:* LISTEN 0 576146 2189/lttng-relayd tcp 0 0 192.168.0.1:56915 0.0.0.0:* LISTEN 0 576147 2189/lttng-relayd tcp 0 0 192.168.0.1:52777 192.168.0.4:56340 ESTABLISHED 0 576207 2189/lttng-relayd tcp 0 0 192.168.0.1:36245 192.168.0.1:52777 ESTABLISHED 0 579913 2170/lttng-consumer tcp 0 0 192.168.0.1:52777 192.168.0.1:36245 ESTABLISHED 0 576208 2189/lttng-relayd tcp 0 0 192.168.0.1:56915 192.168.0.3:46775 ESTABLISHED 0 576206 2189/lttng-relayd tcp 0 0 192.168.0.1:33401 192.168.0.1:56915 ESTABLISHED 0 579225 2170/lttng-consumer tcp 0 0 192.168.0.1:56915 192.168.0.1:33401 ESTABLISHED 0 579226 2189/lttng-relayd tcp 0 0 192.168.0.1:56915 192.168.0.2:56282 ESTABLISHED 0 576210 2189/lttng-relayd tcp 0 0 192.168.0.1:52777 192.168.0.3:55659 ESTABLISHED 0 576205 2189/lttng-relayd tcp 0 0 192.168.0.1:56915 192.168.0.4:36738 ESTABLISHED 0 576211 2189/lttng-relayd tcp 0 0 192.168.0.1:52777 192.168.0.2:37559 ESTABLISHED 0 576209 2189/lttng-relayd Wed Dec 19 10:50:50 EST 2012 Deactivating Profile #################### Wed Dec 19 10:50:50 EST 2012 Dec 19 10:50:50 SC-1 osafimmnd[6451]: Ccb 546 COMMITTED (TraceC) Dec 19 10:50:50 PL-3 osafimmnd[4905]: Ccb 546 COMMITTED (TraceC) Dec 19 10:50:50 PL-4 osafimmnd[4935]: Ccb 546 COMMITTED (TraceC) Dec 19 10:50:50 SC-2 osafimmnd[5121]: Ccb 546 COMMITTED (TraceC) Dec 19 10:50:50 SC-1 osafimmnd[6451]: Ccb 547 COMMITTED (TraceC) Dec 19 10:50:50 PL-3 osafimmnd[4905]: Ccb 547 COMMITTED (TraceC) Dec 19 10:50:50 PL-4 osafimmnd[4935]: Ccb 547 COMMITTED (TraceC) Dec 19 10:50:50 SC-2 osafimmnd[5121]: Ccb 547 COMMITTED (TraceC) Dec 19 10:50:51 SC-1 osafimmnd[6451]: Ccb 548 COMMITTED (TraceC) Dec 19 10:50:51 PL-3 osafimmnd[4905]: Ccb 548 COMMITTED (TraceC) Dec 19 10:50:51 PL-4 osafimmnd[4935]: Ccb 548 COMMITTED (TraceC) Dec 19 10:50:51 SC-2 osafimmnd[5121]: Ccb 548 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 13 [in process_client_msg() at main.c:2204] DEBUG1: Counting number of available session for UID 0 GID 0 [in lttng_sessions_count() at main.c:2175] DEBUG1: Getting all available session for UID 0 GID 0 [in cmd_list_lttng_sessions() at cmd.c:2074] DEBUG1: Sending response (size: 4388, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 17 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG1: Stopping all UST traces [in ust_app_stop_trace_all() at ust-app.c:2544] DEBUG1: Stopping tracing for ust app pid 892 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 893 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 900 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Stopping tracing for ust app pid 891 [in ust_app_stop_trace() at ust-app.c:2386] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 24 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: Consumer data pending for id 1 [in consumer_is_data_pending() at consumer.c:784] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST consumer data pending command for id 1 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:372] DEBUG1: Consumer data pending command on session id 1 [in consumer_data_pending() at consumer.c:2956] DEBUG1: Relayd begin data pending [in relayd_begin_data_pending() at relayd.c:522] DEBUG3: Relayd sending command 10 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 62 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 62 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 63 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 63 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 64 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 64 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 65 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 65 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 67 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 67 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 68 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 68 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 69 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 69 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 70 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 70 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 72 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 72 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 73 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 73 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 74 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 74 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 75 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 75 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd checking quiescent control state [in relayd_quiescent_control() at relayd.c:478] DEBUG3: Relayd sending command 9 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 77 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 77 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 78 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 78 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 79 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 79 [in relayd_data_pending() at relayd.c:459] DEBUG1: UST consumer checking data pending [in lttng_ustconsumer_data_pending() at ust-consumer.c:605] DEBUG1: Relayd data pending for stream id 80 [in relayd_data_pending() at relayd.c:430] DEBUG3: Relayd sending command 8 of size 40 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd data is NOT pending for stream id 80 [in relayd_data_pending() at relayd.c:459] DEBUG1: Relayd end data pending [in relayd_end_data_pending() at relayd.c:569] DEBUG3: Relayd sending command 11 of size 32 [in send_command() at relayd.c:74] DEBUG3: Relayd waiting for reply of size 4 [in recv_reply() at relayd.c:90] DEBUG1: Relayd end data pending is data inflight: 0 [in relayd_end_data_pending() at relayd.c:593] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG1: Consumer data pending ret 0 [in consumer_is_data_pending() at consumer.c:820] DEBUG1: Sending response (size: 16, retcode: Unknown error code) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2204] DEBUG1: Getting session TestProf_1 by name [in process_client_msg() at main.c:2275] DEBUG2: Trying to find session by name TestProf_1 [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:700] DEBUG2: Sending destroy relayd command to consumer... [in consumer_send_destroy_relayd() at consumer.c:79] DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2610] DEBUG1: UST consumer destroying relayd 2 [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:331] DEBUG1: received fds on sock [in consumer_thread_sessiond_poll() at consumer.c:2628] DEBUG2: Consumer send destroy relayd command done [in consumer_send_destroy_relayd() at consumer.c:102] DEBUG1: Destroy all UST traces [in ust_app_destroy_trace_all() at ust-app.c:2570] DEBUG1: Destroy tracing for ust app pid 892 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 9 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG1: Destroy tracing for ust app pid 893 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 9 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG1: Destroy tracing for ust app pid 900 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 9 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG1: Destroy tracing for ust app pid 891 [in destroy_trace() at ust-app.c:2469] DEBUG2: Trace UST destroy metadata 9 [in trace_ust_destroy_metadata() at trace-ust.c:586] DEBUG2: Trace UST destroy session 1 [in trace_ust_destroy_session() at trace-ust.c:668] DEBUG1: Destroying session TestProf_1 [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3329] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] Error: Relayd close stream replied error -1868275200 DEBUG1: Relayd close stream id 66 successfully [in relayd_send_close_stream() at relayd.c:409] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 40 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata poll wait with 4 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2118] DEBUG1: Metadata fd 32 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: In read_subbuffer (wait_fd: 32, stream key: 56) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd sending metadata of size 2403 [in relayd_send_metadata() at relayd.c:287] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2391) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2391) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG3: Consumer delete metadata stream 32 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Relayd closing stream id 71 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 54 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata fd 37 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: In read_subbuffer (wait_fd: 37, stream key: 70) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd sending metadata of size 2401 [in relayd_send_metadata() at relayd.c:287] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2389) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2389) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG3: Consumer delete metadata stream 37 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Relayd closing stream id 76 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 68 [in consumer_del_channel() at consumer.c:822] DEBUG1: Metadata fd 22 is hup|err. [in consumer_thread_metadata_poll() at consumer.c:2205] DEBUG1: Attempting to flush and consume the UST buffers [in consumer_thread_metadata_poll() at consumer.c:2209] DEBUG1: In read_subbuffer (wait_fd: 22, stream key: 28) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Relayd sending metadata of size 2848 [in relayd_send_metadata() at relayd.c:287] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2836) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2836) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG3: Consumer delete metadata stream 22 [in consumer_del_metadata_stream() at consumer.c:1835] DEBUG1: Relayd closing stream id 61 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_metadata_stream() at consumer.c:1909] DEBUG1: Consumer delete channel key 26 [in consumer_del_channel() at consumer.c:822] DEBUG1: polling on 17 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 16 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 28 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 28, stream key: 44) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Normal read on fd 35 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: Metadata poll wait with 1 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2116] DEBUG1: In read_subbuffer (wait_fd: 35, stream key: 62) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Connection reset by peer [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 35 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 74 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 33 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 33, stream key: 58) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 33 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 72 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 31 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 31, stream key: 50) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 31 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 70 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 36 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 36, stream key: 64) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 36 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 75 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 30 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 30, stream key: 48) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Normal read on fd 34 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 34, stream key: 60) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 34 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 73 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 52 [in consumer_del_channel() at consumer.c:822] DEBUG1: Normal read on fd 29 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 29, stream key: 46) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG1: Reserving sub buffer failed (everything is normal, it is due to concurrency) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:517] DEBUG1: Polling fd 28 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 28 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 67 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: fd 25 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 25 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 40 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 40 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Polling fd 30 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 30 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 69 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: fd 23 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 23 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 24 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 24 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 38 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 38 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 39 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 39 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: fd 41 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 41 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Polling fd 29 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Consumer del stream 29 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 68 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 38 [in consumer_del_channel() at consumer.c:822] DEBUG1: fd 26 is hup|err|nval. Attempting flush and read. [in consumer_thread_data_poll() at consumer.c:2462] DEBUG1: Polling fd 26 tells it has hung up. [in consumer_thread_data_poll() at consumer.c:2473] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: Active FD 25 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 40 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 23 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 24 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 38 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 39 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 41 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: Active FD 26 [in consumer_update_poll_array() at consumer.c:980] DEBUG1: polling on 9 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG1: poll num_rdy : 8 [in consumer_thread_data_poll() at consumer.c:2343] DEBUG1: Normal read on fd 25 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 25, stream key: 34) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 2417) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 2417) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 25 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 64 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 40 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 40, stream key: 76) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 40 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 79 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 23 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 23, stream key: 30) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 207) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 207) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 23 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 62 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 24 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 24, stream key: 32) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 24 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 63 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 38 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 38, stream key: 72) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 38 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 77 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 39 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 39, stream key: 74) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 39 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 78 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Normal read on fd 41 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 41, stream key: 78) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 68) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 68) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 41 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 80 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer delete channel key 66 [in consumer_del_channel() at consumer.c:822] DEBUG1: Normal read on fd 26 [in consumer_thread_data_poll() at consumer.c:2439] DEBUG1: In read_subbuffer (wait_fd: 26, stream key: 36) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:489] DEBUG3: Relayd sending data header of size 32 [in relayd_send_data_hdr() at relayd.c:345] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Consumer mmap write() ret -1 (len 3782) [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1436] DEBUG1: Error in file write mmap [in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1444] DEBUG1: Error writing to tracefile (ret: -1 != len: 4096 != subbuf_size: 3782) [in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:551] DEBUG1: Consumer del stream 26 [in consumer_del_stream() at consumer.c:401] DEBUG1: Relayd closing stream id 65 [in relayd_send_close_stream() at relayd.c:381] PERROR: sendmsg inet: Broken pipe [in lttcomm_sendmsg_inet_sock() at inet.c:279] DEBUG1: Unable to close stream on the relayd. Continuing [in consumer_del_stream() at consumer.c:477] DEBUG1: Consumer destroy and close relayd socket pair [in destroy_relayd() at consumer.c:240] DEBUG1: Consumer delete channel key 24 [in consumer_del_channel() at consumer.c:822] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:964] DEBUG1: polling on 1 fd [in consumer_thread_data_poll() at consumer.c:2341] DEBUG2: Trace destroy UST event com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA [in trace_ust_destroy_event() at trace-ust.c:510] DEBUG3: Relayd closing socket 18 [in relayd_close() at relayd.c:328] DEBUG3: Relayd closing socket 20 [in relayd_close() at relayd.c:328] 6856 10:50:50 12/19/2012 IN TraceC "DBG: Deactivating 6857 10:50:50 12/19/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProf_1,traceMId=1" from Registries 6858 10:50:50 12/19/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProf_1,traceMId=1" is valid 6859 10:50:50 12/19/2012 NO TraceC "Deactivating session TestProf_1. 6860 10:50:50 12/19/2012 IN TraceC "DBG: update Async for deactivate - first time 6861 10:50:50 12/19/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 6862 10:50:50 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x6B04C0 6863 10:50:50 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x6B05F0 6864 10:50:50 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x6B0720 6865 10:50:50 12/19/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x6B0850 6866 10:50:50 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6867 10:50:50 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6868 10:50:50 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B04C0 6869 10:50:50 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK 6870 10:50:50 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B05F0 6871 10:50:50 12/19/2012 IN TraceP_PL-3 "Deactivating 6872 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: hasSession: Invoke lttng_list_session. 6873 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6874 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6875 10:50:50 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B0720 6876 10:50:50 12/19/2012 IN TraceP_PL-4 "Deactivating 6877 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: hasSession: Invoke lttng_list_session. 6878 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6879 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6880 10:50:50 12/19/2012 IN TraceC "DBG: Invoking admin op 0x6B0850 6881 10:50:50 12/19/2012 IN TraceP_SC-1 "Deactivating 6882 10:50:50 12/19/2012 IN TraceP_SC-1 "DBG: hasSession: Invoke lttng_list_session. 6883 10:50:50 12/19/2012 IN TraceP_SC-1 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6884 10:50:50 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6885 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6886 10:50:50 12/19/2012 IN TraceP_SC-2 "Deactivating 6887 10:50:50 12/19/2012 IN TraceP_SC-2 "DBG: hasSession: Invoke lttng_list_session. 6888 10:50:50 12/19/2012 IN TraceP_SC-2 "DBG: hasSession: Reply recieved from lttng_list_session with count 1 6889 10:50:50 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Invoke lttng_stop_tracing for session TestProf_1 6890 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6891 10:50:50 12/19/2012 IN TraceP_PL-3 "DBG: Return for invocation -13 value: 1, return code: 1 , SA_AIS_OK 6892 10:50:50 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6893 10:50:50 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6894 10:50:50 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6895 10:50:50 12/19/2012 IN TraceP_PL-3 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6896 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6897 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6898 10:50:50 12/19/2012 IN TraceP_PL-4 "DBG: Return for invocation -14 value: 1, return code: 1 , SA_AIS_OK 6899 10:50:50 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6900 10:50:50 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6901 10:50:50 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6902 10:50:50 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6903 10:50:50 12/19/2012 IN TraceP_PL-4 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6904 10:50:51 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Invoke lttng_destroy_session TestProf_1. 6905 10:50:51 12/19/2012 IN TraceP_SC-2 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6906 10:50:51 12/19/2012 IN TraceP_SC-2 "DBG: Return for invocation -16 value: 1, return code: 1 , SA_AIS_OK 6907 10:50:51 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6908 10:50:51 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6909 10:50:51 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6910 10:50:51 12/19/2012 IN TraceP_SC-2 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6911 10:50:51 12/19/2012 IN TraceP_SC-1 "DBG: deactivateSession: Reply recieved for lttng_destroy_session for session TestProf_1 6912 10:50:51 12/19/2012 IN TraceP_SC-1 "DBG: Return for invocation -15 value: 1, return code: 1 , SA_AIS_OK 6913 10:50:51 12/19/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 6914 10:50:51 12/19/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 6915 10:50:51 12/19/2012 IN TraceC "receiveTracePDeactivateResult result: 1 op_result 1 6916 10:50:51 12/19/2012 IN TraceC "deactivateSessionOnTracePsComplete 6917 10:50:51 12/19/2012 IN TraceC "DBG: Terminating relayD for session TestProf_1 6918 10:50:51 12/19/2012 IN TraceP_SC-1 "Deactivation was completed successfully for session traceSessionRecordingId=TestProf_1,traceMId=1. 6919 10:50:51 12/19/2012 NO TraceC "Deactivate complete for session "TestProf_1" 6920 10:50:51 12/19/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProf_1,traceMId=1 6921 10:50:51 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6922 10:50:51 12/19/2012 IN TraceC "DBG: Received death of pid 2189 with ret 9. 6923 10:50:51 12/19/2012 IN TraceC "DBG: No handler for pid 2189. 6924 10:50:51 12/19/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProf,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Dec 19 10:50:51 EST 2012 Printing Profile #################### Wed Dec 19 10:50:51 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 12/19/12T10:50:51-0500 progress.timeActionStarted : 12/19/12T10:50:50-0500 progress.timeOfLastStatusUpdate : 12/19/12T10:50:51-0500 Wed Dec 19 10:50:51 EST 2012 Printing Session #################### Wed Dec 19 10:50:51 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:50:39-0500 state : COMPLETE stopTime : 12/19/12T10:50:51-0500 traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress : "" 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-105041 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-105041 20K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1 16K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-105039 12K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-105039 48K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust 52K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-105039 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-105039 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-105039 20K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-105039 100K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust 104K /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-105039 8.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-105039 4.0K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-105039 24K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust 28K /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2 216K /storage/no-backup/tracecc-apr9010500/TestProf_1 Wed Dec 19 10:50:51 EST 2012 Printing Ports after Session has been deactivated #################### Wed Dec 19 10:50:51 EST 2012 Converting Session #################### Wed Dec 19 10:50:51 EST 2012 6925 10:50:51 12/19/2012 IN TraceC "DBG: Converting output operation 6926 10:50:51 12/19/2012 IN TraceC "DBG: Try to async-remove directory "/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs" 6927 10:50:51 12/19/2012 IN TraceC "DBG: Directory /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs was created successfully! 6928 10:50:51 12/19/2012 NO TraceC "Converting session "TestProf_1" 6929 10:50:51 12/19/2012 IN TraceC "DBG: resetOrCreate traceSessionRecordingId=TestProf_1,traceMId=1 6930 10:50:51 12/19/2012 IN TraceC "DBG: ConvertAsyncProgress: Create request for child object of traceSessionRecordingId=TestProf_1,traceMId=1 6931 10:50:51 12/19/2012 IN TraceC "DBG: ConvertAsyncProgressImplementer: id=1,traceSessionRecordingId=TestProf_1,traceMId=1 was successfully created 6932 10:50:51 12/19/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 6933 10:50:51 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/PL-3 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-3.log ==- 6934 10:50:51 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/PL-3,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-3.log === 6935 10:50:51 12/19/2012 IN TraceC "DBG: processMostQueuedEvents : directory Converted_logs is being added to parent directory /storage/no-backup/tracecc-apr9010500/TestProf_1 6936 10:50:54 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6937 10:50:54 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6938 10:50:54 12/19/2012 IN TraceC "DBG: Received death of pid 2291 with ret 256. 6939 10:50:54 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6940 10:50:54 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6941 10:50:54 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6942 10:50:54 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/PL-4 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-4.log ==- 6943 10:50:54 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/PL-4,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_PL-4.log === 6944 10:50:55 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6945 10:50:55 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6946 10:50:55 12/19/2012 IN TraceC "DBG: Received death of pid 2295 with ret 256. 6947 10:50:55 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6948 10:50:55 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6949 10:50:55 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6950 10:50:55 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/SC-1 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-1.log ==- 6951 10:50:55 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/SC-1,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-1.log === 6952 10:50:55 12/19/2012 IN TraceC "DBG: Dispatch: epoll_wait returned an error: 4 6953 10:50:55 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6954 10:50:55 12/19/2012 IN TraceC "DBG: Received death of pid 2297 with ret 256. 6955 10:50:55 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6956 10:50:55 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6957 10:50:55 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6958 10:50:55 12/19/2012 IN TraceC "DBG: -== start convert output from /storage/no-backup/tracecc-apr9010500//TestProf_1/SC-2 to /storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-2.log ==- 6959 10:50:55 12/19/2012 IN TraceC "DBG: === ConvertOutputTask::babelTraceConvert, bfile=/storage/no-backup/tracecc-apr9010500//TestProf_1/SC-2,desFile=/storage/no-backup/tracecc-apr9010500//TestProf_1/Converted_logs/Node_SC-2.log === 6960 10:50:55 12/19/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 6961 10:50:55 12/19/2012 IN TraceC "DBG: Received death of pid 2299 with ret 256. 6962 10:50:55 12/19/2012 IN TraceC "DBG: ConvertOutputTask::handleSigChild line=61 return: 1 6963 10:50:55 12/19/2012 IN TraceC "DBG: receiveConvertOutputResult() result: 1 6964 10:50:55 12/19/2012 ER TraceC "Convert Session: TestProf_1 session log conversion failed 6965 10:50:55 12/19/2012 NO TraceC "Completed conversion of session "TestProf_1". DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Health state current 598, last 100, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] Wed Dec 19 10:50:55 EST 2012 Listing Session Directory Structure #################### /storage/no-backup/tracecc-apr9010500/TestProf_1: total 32 drwxr-xr-x 7 root root 4096 Dec 19 10:50 . drwxr-xr-x 5 root root 4096 Dec 19 10:50 .. drwxr-xr-x 2 root root 4096 Dec 19 10:50 Converted_logs drwxr-x--- 3 root root 4096 Dec 19 10:50 PL-3 drwxr-x--- 3 root root 4096 Dec 19 10:50 PL-4 drwxr-x--- 3 root root 4096 Dec 19 10:50 SC-1 drwxr-x--- 3 root root 4096 Dec 19 10:50 SC-2 -rw-r--r-- 1 root root 230 Dec 19 10:50 session_metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/Converted_logs: total 48 drwxr-xr-x 2 root root 4096 Dec 19 10:50 . drwxr-xr-x 7 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 26814 Dec 19 10:50 Node_PL-3.log -rwxr-xr-x 1 root root 0 Dec 19 10:50 Node_PL-4.log -rwxr-xr-x 1 root root 0 Dec 19 10:50 Node_SC-1.log -rwxr-xr-x 1 root root 0 Dec 19 10:50 Node_SC-2.log -rw-r--r-- 1 root root 9341 Dec 19 10:50 conversionResult.log /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:50 . drwxr-xr-x 7 root root 4096 Dec 19 10:50 .. drwxr-x--- 6 root root 4096 Dec 19 10:50 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:50 . drwxr-x--- 3 root root 4096 Dec 19 10:50 .. drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-13775-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-13783-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Thread1-13776-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_type1-13774-20121219-105039 /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13775-20121219-105039: total 24 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Fork1-13783-20121219-105039: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_Thread1-13776-20121219-105039: total 28 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-3/ust/TestApp_type1-13774-20121219-105039: total 32 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 8192 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:50 . drwxr-xr-x 7 root root 4096 Dec 19 10:50 .. drwxr-x--- 6 root root 4096 Dec 19 10:50 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:50 . drwxr-x--- 3 root root 4096 Dec 19 10:50 .. drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-13832-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-13841-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Thread1-13833-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_type1-13831-20121219-105039 /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13832-20121219-105039: total 12 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Fork1-13841-20121219-105039: total 20 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_Thread1-13833-20121219-105039: total 12 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/PL-4/ust/TestApp_type1-13831-20121219-105039: total 16 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:50 . drwxr-xr-x 7 root root 4096 Dec 19 10:50 .. drwxr-x--- 6 root root 4096 Dec 19 10:50 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:50 . drwxr-x--- 3 root root 4096 Dec 19 10:50 .. drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-892-20121219-105041 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-900-20121219-105041 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Thread1-893-20121219-105041 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_type1-891-20121219-105041 /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-892-20121219-105041: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Fork1-900-20121219-105041: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_Thread1-893-20121219-105041: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-1/ust/TestApp_type1-891-20121219-105041: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2: total 12 drwxr-x--- 3 root root 4096 Dec 19 10:50 . drwxr-xr-x 7 root root 4096 Dec 19 10:50 .. drwxr-x--- 6 root root 4096 Dec 19 10:50 ust /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust: total 24 drwxr-x--- 6 root root 4096 Dec 19 10:50 . drwxr-x--- 3 root root 4096 Dec 19 10:50 .. drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-16605-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Fork1-16613-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_Thread1-16606-20121219-105039 drwxr-x--- 2 root root 4096 Dec 19 10:50 TestApp_type1-16604-20121219-105039 /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16605-20121219-105039: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Fork1-16613-20121219-105039: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_Thread1-16606-20121219-105039: total 8 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata /storage/no-backup/tracecc-apr9010500/TestProf_1/SC-2/ust/TestApp_type1-16604-20121219-105039: total 12 drwxr-x--- 2 root root 4096 Dec 19 10:50 . drwxr-x--- 6 root root 4096 Dec 19 10:50 .. -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_0 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_1 -rwxr-xr-x 1 root root 4096 Dec 19 10:50 channel0_2 -rwxr-xr-x 1 root root 0 Dec 19 10:50 channel0_3 -rwxr-xr-x 1 root root 0 Dec 19 10:50 metadata Wed Dec 19 10:50:55 EST 2012 Printing Session #################### Wed Dec 19 10:50:55 EST 2012 errors : NoError; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 12/19/12T10:50:39-0500 state : COMPLETE stopTime : 12/19/12T10:50:51-0500 traceExpressions : com_ericsson_cba_trace_testapp_lowtraf:TenPerSecA,TRACE_DEBUG_MODULE(10) traceProfileId : TestProf -------------------------------- progress.actionName : convertOutput progress.additionalInfo : nodeName=All processId=All progress.info : Convert is terminated with failure! progress.percentage : 100 progress.result : FAILURE progress.resultInfo : Failed to convert output progress.state : FINISHED progress.timeActionCompleted : 12/19/12T10:50:55-0500 progress.timeActionStarted : 12/19/12T10:50:51-0500 progress.timeOfLastStatusUpdate : "" Wed Dec 19 10:50:55 EST 2012 Deleting Session #################### Wed Dec 19 10:50:55 EST 2012 Deleting Profile #################### Wed Dec 19 10:50:55 EST 2012 Dec 19 10:50:56 SC-1 osafimmnd[6451]: Ccb 553 COMMITTED (TraceCLI) Dec 19 10:50:56 PL-3 osafimmnd[4905]: Ccb 553 COMMITTED (TraceCLI) Dec 19 10:50:56 PL-4 osafimmnd[4935]: Ccb 553 COMMITTED (TraceCLI) Dec 19 10:50:56 SC-2 osafimmnd[5121]: Ccb 553 COMMITTED (TraceCLI) ########################################################################### ########################################################################### ########################################################################### SC-1:/storage/no-backup/coremw/var/log/saflog # DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3233] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3278] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3282] DEBUG1: Health state current 606, last 100, ret 0 [in health_check_state() at health.c:127] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:476] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3191] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 33, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3041] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 1, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG1: Health state current 15, last 15, ret 0 [in health_check_state() at health.c:127] DEBUG3: Health consumer check 1 [in check_consumer_health() at main.c:1941] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3094] DEBUG1: Health check ready [in thread_manage_health() at main.c:2992]