SC-1:/storage/no-backup/coremw/var/log/saflog # ps -eaf | egrep "trace|lttng" | grep -v grep pkill -9 lttng-sessiond; usleep 100000; lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 & sleep 2;ps -eaf | egrep "trace|lttng" | grep -v grep /cluster/temp/eusgarc/test root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19548 1 0 11:30 ? 00:00:00 lttng-sessiond -d root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p SC-1:/storage/no-backup/coremw/var/log/saflog # pkill -9 lttng-sessiond; usleep 100000; lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 & [4] 25466 SC-1:/storage/no-backup/coremw/var/log/saflog # sleep 2;ps -eaf | egrep "trace|lttng" | grep -v grep DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:3662] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:3910] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:3912] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:3965] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:3966] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:3967] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:3976] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:3978] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:3987] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:3989] DEBUG3: Created hashtable size 4 at 0x6420a0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x6425c0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:3704] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:3704] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:3704] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:3796] DEBUG3: Session daemon client socket 8 and application socket 9 created [in init_daemon_socket() at main.c:3566] Warning: No tracing group detected DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:224] DEBUG1: Command subsystem initialized [in cmd_init() at cmd.c:2460] DEBUG1: [thread] Manage health check started [in thread_manage_health() at main.c:2928] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3122] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1284] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1294] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1095] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1360] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1124] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:497] DEBUG1: Got the wait shm fd 17 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:63] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1393] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Health state current 12, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3253] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Health state current 5, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Health state current 1, last 0, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2181] DEBUG1: Waiting for 1 URIs from client ... [in process_client_msg() at main.c:2689] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG1: Tracing session _Trace_EA created in (null) with ID 0 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x64db90 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting trace directory path from URI to /tmp/ [in add_uri_to_consumer() at cmd.c:407] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2181] DEBUG1: Getting session _Trace_EA by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:730] DEBUG1: Destroying session _Trace_EA [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25466 10405 0 12:33 pts/2 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel SC-1:/storage/no-backup/coremw/var/log/saflog # /cluster/temp/eusgarc/test Wed Nov 14 12:33:38 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25466 10405 0 12:33 pts/2 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel Wed Nov 14 12:33:38 EST 2012 Creating Profile 1 #################### Wed Nov 14 12:33:38 EST 2012 Nov 14 12:33:38 SC-1 osafimmnd[6429]: Ccb 3867 COMMITTED (TraceCLI) Wed Nov 14 12:33:38 EST 2012 Wait 1 sec... Nov 14 12:33:38 SC-1 osafimmnd[6429]: Ccb 3868 COMMITTED (TraceC) DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2181] DEBUG1: Waiting for 1 URIs from client ... [in process_client_msg() at main.c:2689] DEBUG2: Trying to find session by name /_TraceCValidator_JhYHUV [in session_find_by_name() at session.c:122] DEBUG1: Tracing session /_TraceCValidator_JhYHUV created in (null) with ID 1 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name /_TraceCValidator_JhYHUV [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x64db90 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting trace directory path from URI to /tmp/_TraceCValidator_JhYHUV [in add_uri_to_consumer() at cmd.c:407] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2181] DEBUG1: Getting session /_TraceCValidator_JhYHUV by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name /_TraceCValidator_JhYHUV [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2068] DEBUG3: Created hashtable size 4 at 0x6469d0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x646f10 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x647450 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x653110 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:143] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2020] DEBUG3: Created hashtable size 4 at 0x653110 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir /ust [in copy_session_consumer() at main.c:2038] DEBUG1: Spawning consumerd [in spawn_consumerd() at main.c:1701] DEBUG2: Consumer pid 25488 [in start_consumerd() at main.c:1882] DEBUG2: Spawning consumer control thread [in start_consumerd() at main.c:1885] DEBUG1: [thread] Manage consumer started [in thread_manage_consumer() at main.c:845] DEBUG1: Using 64-bit UST consumer at: /usr/lib64/lttng/libexec/lttng-consumerd [in spawn_consumerd() at main.c:1775] DEBUG1: epoll set max size is 403353 [in compat_epoll_set_max_size() at compat-epoll.c:224] DEBUG3: Created hashtable size 4 at 0x61f030 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x61f570 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x61fab0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x61fff0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x620530 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Connecting to error socket /var/run/lttng/ustconsumerd64/error [in main() at lttng-consumerd.c:358] DEBUG1: Thread metadata poll started [in consumer_thread_metadata_poll() at consumer.c:2000] DEBUG3: Created hashtable size 4 at 0x620ec0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: Metadata main loop started [in consumer_thread_metadata_poll() at consumer.c:2015] DEBUG1: Metadata poll wait with 1 fd(s) [in consumer_thread_metadata_poll() at consumer.c:2028] DEBUG1: Creating command socket /var/run/lttng/ustconsumerd64/command [in consumer_thread_sessiond_poll() at consumer.c:2447] DEBUG1: Updating poll fd array [in consumer_update_poll_array() at consumer.c:901] DEBUG1: polling on 1 fd [in consumer_thread_data_poll() at consumer.c:2249] DEBUG1: Sending ready command to lttng-sessiond [in consumer_thread_sessiond_poll() at consumer.c:2460] DEBUG1: Connection on client_socket [in consumer_thread_sessiond_poll() at consumeDEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:141] DEBUG1: Enabling channel channel0 for session /_TraceCValidator_JhYHUV [in cmd_enable_channel() at cmd.c:822] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:52] DEBUG3: Created hashtable size 4 at 0x654ac0 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x655000 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:207] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:273] DEBUG2: UST app adding channel channel0 to global domain for session id 1 [in ust_app_create_channel_glb() at ust-app.c:1994] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:298] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2181] DEBUG1: Getting session /_TraceCValidator_JhYHUV by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name /_TraceCValidator_JhYHUV [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:730] DEBUG1: Destroy all UST traces [in ust_app_destroy_trace_all() at ust-app.c:2514] DEBUG2: Trace UST destroy session 1 [in trace_ust_destroy_session() at trace-ust.c:576] DEBUG1: Destroying session /_TraceCValidator_JhYHUV [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG2: Trace destroy UST channel channel0 [in trace_ust_destroy_channel() at trace-ust.c:459] 12463 12:33:38 11/14/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 12464 12:33:38 11/14/2012 IN TraceC "DBG: newvalue size=27 12465 12:33:38 11/14/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProfile1 12466 12:33:38 11/14/2012 IN TraceC "DBG: ProfileAdmin::traceExpressionsValidator was called 12467 12:33:38 11/14/2012 IN TraceC "DBG: traceExpressions size=2 12468 12:33:38 11/14/2012 IN TraceC "DBG: About to create a temporary session /_TraceCValidator_JhYHUV in path /tmp/_TraceCValidator_JhYHUV 12469 12:33:38 11/14/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_JhYHUV" 12470 12:33:38 11/14/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_JhYHUV" Deleted! 12471 12:33:38 11/14/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProfile1,traceMId=1 Wed Nov 14 12:33:39 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25466 10405 0 12:33 pts/2 00:00:00 lttng-sessiond -vvv --verbose-consumer --no-kernel root 25488 25466 0 12:33 pts/2 00:00:00 lttng-consumerd --verbose -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error Wed Nov 14 12:33:39 EST 2012 Activating Profile 1 #################### Wed Nov 14 12:33:39 EST 2012 Nov 14 12:33:39 SC-1 osafimmnd[6429]: Ccb 3870 COMMITTED (TraceC) 12472 12:33:39 11/14/2012 IN TraceC "DBG: Activating 12473 12:33:39 11/14/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProfile1_1,traceMId=1 as a runtime object 12474 12:33:39 11/14/2012 NO TraceC "Activating Session TestProfile1_1. 12475 12:33:39 11/14/2012 IN TraceC "DBG: Directory /home/trace//TestProfile1_1 was created successfully! 12476 12:33:39 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12477 12:33:39 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12478 12:33:39 11/14/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 59383 12479 12:33:39 11/14/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 50886 12480 12:33:39 11/14/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 12481 12:33:39 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12482 12:33:39 11/14/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProfile1_1" pid: 25505 bound on 192.168.0.1:59383:50886 12483 12:33:39 11/14/2012 IN TraceC "DBG: Done Activating 12484 12:33:39 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile1_1 is being added to parent directory /home/trace 12485 12:33:39 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:33:40 SC-1 osafimmnd[6429]: Ccb 3871 COMMITTED (TraceC) 12486 12:33:40 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK DEBUG1: UST registration received with pid:25213 ppid:25202 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1493] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1393] DEBUG1: UST registration received with pid:25203 ppid:10361 uid:0 gid:0 sock:21 name:TestApp_Thread1 (version 2.0) [in thread_registration_apps() at main.c:1493] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1393] DEBUG1: UST registration received with pid:25202 ppid:10361 uid:0 gid:0 sock:22 name:TestApp_Fork1 (version 2.0) [in thread_registration_apps() at main.c:1493] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1393] DEBUG1: UST registration received with pid:25204 ppid:10361 uid:0 gid:0 sock:23 name:TestApp_type1 (version 2.0) [in thread_registration_apps() at main.c:1493] DEBUG1: Futex n to 1 wake done [in futex_nto1_wake() at futex.c:104] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1393] DEBUG1: Futex n to 1 wait done [in futex_nto1_wait() at futex.c:90] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:75] DEBUG1: Dispatching UST registration pid:25213 ppid:25202 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1306] DEBUG1: Dispatching UST registration pid:25203 ppid:10361 uid:0 gid:0 sock:21 name:TestApp_Thread1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1306] DEBUG1: Dispatching UST registration pid:25202 ppid:10361 uid:0 gid:0 sock:22 name:TestApp_Fork1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1306] DEBUG1: Dispatching UST registration pid:25204 ppid:10361 uid:0 gid:0 sock:23 name:TestApp_type1 (version 2.0) [in thread_dispatch_ust_registration() at main.c:1306] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1294] DEBUG3: Created hashtable size 4 at 0x646b40 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:25213 ppid:25202 uid:0 gid:0 sock:17 name:TestApp_Fork1 (version 2.0) [in ust_app_register() at ust-app.c:1467] DEBUG2: UST app PID 25213 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2945] DEBUG1: Apps with sock 17 added to poll set [in thread_manage_apps() at main.c:1221] DEBUG1: Apps thread polling on 3 fds [in thread_manage_apps() at main.c:1124] DEBUG3: Created hashtable size 4 at 0x64db90 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:25203 ppid:10361 uid:0 gid:0 sock:21 name:TestApp_Thread1 (version 2.0) [in ust_app_register() at ust-app.c:1467] DEBUG2: UST app PID 25203 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2945] DEBUG1: Apps with sock 21 added to poll set [in thread_manage_apps() at main.c:1221] DEBUG1: Apps thread polling on 4 fds [in thread_manage_apps() at main.c:1124] DEBUG3: Created hashtable size 4 at 0x654ac0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:25202 ppid:10361 uid:0 gid:0 sock:22 name:TestApp_Fork1 (version 2.0) [in ust_app_register() at ust-app.c:1467] DEBUG2: UST app PID 25202 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2945] DEBUG1: Apps with sock 22 added to poll set [in thread_manage_apps() at main.c:1221] DEBUG1: Apps thread polling on 5 fds [in thread_manage_apps() at main.c:1124] DEBUG3: Created hashtable size 4 at 0x655000 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG1: App registered with pid:25204 ppid:10361 uid:0 gid:0 sock:23 name:TestApp_type1 (version 2.0) [in ust_app_register() at ust-app.c:1467] DEBUG2: UST app PID 25204 is compatible with internal major version 3 (supporting == 3) [in ust_app_validate_version() at ust-app.c:2945] DEBUG1: Apps with sock 23 added to poll set [in thread_manage_apps() at main.c:1221] DEBUG1: Apps thread polling on 6 fds [in thread_manage_apps() at main.c:1124] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3253] DEBUG1: Health state current 120, last 12, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Health state current 53, last 5, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Receiving data from client for health... [in thread_manage_health() at main.c:3016] DEBUG1: Health state current 41, last 1, ret 0 [in health_check_state() at health.c:127] DEBUG2: Health check return value 0 [in thread_manage_health() at main.c:3069] DEBUG1: Health check ready [in thread_manage_health() at main.c:2967] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2181] DEBUG1: Waiting for 1 URIs from client ... [in process_client_msg() at main.c:2689] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG1: Tracing session _Trace_EA created in (null) with ID 2 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x647330 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting trace directory path from URI to /tmp/ [in add_uri_to_consumer() at cmd.c:407] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 9 [in process_client_msg() at main.c:2181] DEBUG1: Getting session _Trace_EA by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name _Trace_EA [in session_find_by_name() at session.c:122] DEBUG3: No kernel session when tearing down session [in kernel_destroy_session() at kernel.c:730] DEBUG1: Destroying session _Trace_EA [in session_destroy() at session.c:149] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:2181] DEBUG1: Waiting for 2 URIs from client ... [in process_client_msg() at main.c:2689] DEBUG2: Trying to find session by name TestProfile1_1 [in session_find_by_name() at session.c:122] DEBUG1: Tracing session TestProfile1_1 created in (null) with ID 3 by UID 0 GID 0 [in session_create() at session.c:236] DEBUG2: Trying to find session by name TestProfile1_1 [in session_find_by_name() at session.c:122] DEBUG3: Created hashtable size 4 at 0x647330 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer control URI set with port 59383 [in consumer_set_network_uri() at consumer.c:382] DEBUG3: Consumer set network uri subdir path SC-1/./ [in consumer_set_network_uri() at consumer.c:434] DEBUG3: Append domain trace name to subdir SC-1/./ [in add_uri_to_consumer() at cmd.c:402] DEBUG2: Setting network URI to consumer [in add_uri_to_consumer() at cmd.c:375] DEBUG3: Consumer data URI set with port 50886 [in consumer_set_network_uri() at consumer.c:391] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 5 [in process_client_msg() at main.c:2181] DEBUG1: Getting session TestProfile1_1 by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name TestProfile1_1 [in session_find_by_name() at session.c:122] DEBUG1: Creating UST session [in create_ust_session() at main.c:2068] DEBUG3: Created hashtable size 4 at 0x64c9f0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64cf30 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x64d470 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x653110 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:143] DEBUG3: Copying tracing session consumer output in UST session [in copy_session_consumer() at main.c:2020] DEBUG3: Created hashtable size 4 at 0x653110 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Copy session consumer subdir SC-1/.//ust [in copy_session_consumer() at main.c:2038] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:141] DEBUG1: Enabling channel channel0 for session TestProfile1_1 [in cmd_enable_channel() at cmd.c:822] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:52] DEBUG3: Created hashtable size 4 at 0x656990 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x656e40 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:207] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:273] DEBUG2: UST app adding channel channel0 to global domain for session id 3 [in ust_app_create_channel_glb() at ust-app.c:1994] DEBUG2: UST app pid: 25204 session id 3 not found, creating it [in create_ust_app_session() at ust-app.c:961] DEBUG3: Created hashtable size 4 at 0x658350 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:870] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:988] DEBUG3: Created hashtable size 4 at 0x6595b0 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x659850 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:302] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:810] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:847] DEBUG2: UST app channel channel0 created successfully for pid:25204 and sock:23 [in create_ust_channel() at ust-app.c:683] DEBUG2: UST app create channel channel0 for PID 25204 completed [in create_ust_app_channel() at ust-app.c:1233] DEBUG2: UST app pid: 25202 session id 3 not found, creating it [in create_ust_app_session() at ust-app.c:961] DEBUG3: Created hashtable size 4 at 0x65ad60 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:870] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:988] DEBUG3: Created hashtable size 4 at 0x65b530 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65ba70 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:302] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:810] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:847] DEBUG2: UST app channel channel0 created successfully for pid:25202 and sock:22 [in create_ust_channel() at ust-app.c:683] DEBUG2: UST app create channel channel0 for PID 25202 completed [in create_ust_app_channel() at ust-app.c:1233] DEBUG2: UST app pid: 25203 session id 3 not found, creating it [in create_ust_app_session() at ust-app.c:961] DEBUG3: Created hashtable size 4 at 0x65d150 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:870] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:988] DEBUG3: Created hashtable size 4 at 0x65d940 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x65de80 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:302] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:810] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:847] DEBUG2: UST app channel channel0 created successfully for pid:25203 and sock:21 [in create_ust_channel() at ust-app.c:683] DEBUG2: UST app create channel channel0 for PID 25203 completed [in create_ust_app_channel() at ust-app.c:1233] DEBUG2: UST app pid: 25213 session id 3 not found, creating it [in create_ust_app_session() at ust-app.c:961] DEBUG3: Created hashtable size 4 at 0x65f560 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Shadow copy of session handle -1 [in shadow_copy_session() at ust-app.c:870] DEBUG2: UST app session created successfully with handle 1 [in create_ust_app_session() at ust-app.c:988] DEBUG3: Created hashtable size 4 at 0x65fd50 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x660290 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app channel channel0 allocated [in alloc_ust_app_channel() at ust-app.c:302] DEBUG2: UST app shadow copy of channel channel0 started [in shadow_copy_channel() at ust-app.c:810] DEBUG3: UST app shadow copy of channel channel0 done [in shadow_copy_channel() at ust-app.c:847] DEBUG2: UST app channel channel0 created successfully for pid:25213 and sock:17 [in create_ust_channel() at ust-app.c:683] DEBUG2: UST app create channel channel0 for PID 25213 completed [in create_ust_app_channel() at ust-app.c:1233] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:298] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3300] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:477] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3162] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3204] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3249] DEBUG1: Processing client command 6 [in process_client_msg() at main.c:2181] DEBUG1: Getting session TestProfile1_1 by name [in process_client_msg() at main.c:2252] DEBUG2: Trying to find session by name TestProfile1_1 [in session_find_by_name() at session.c:122] DEBUG3: Consumer socket created (fd: 20) and added to output [in consumer_create_socket() at consumer.c:141] DEBUG2: Trace UST channel channel0 found by name [in trace_ust_find_channel_by_name() at trace-ust.c:47] DEBUG2: Trace UST event NOT found by name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec [in trace_ust_find_event_by_name() at trace-ust.c:79] DEBUG3: Created hashtable size 4 at 0x660b90 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST event com_ericsson_cba_trace_testapp_hightraf:HundredPerSec, loglevel (1,7) created [in trace_ust_create_event() at trace-ust.c:286] DEBUG1: UST app creating event com_ericsson_cba_trace_testapp_hightraf:HundredPerSec for all apps for session id 3 [in ust_app_create_event_glb() at ust-app.c:2119] DEBUG3: Created hashtable size 4 at 0x661460 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: UST app event com_ericsson_cba_trace_testapp_hightraf:HundredPerSec allocated [in alloc_ust_app_event() at ust-app.c:337] Error: Error ustctl create event com_ericsson_cba_trace_testapp_hightraf:HundredPerSec for app pid: 25204 with ret -1028 lttng-sessiond: ust-app.c:1277: create_ust_app_event: Assertion `ret == -LTTNG_UST_ERR_EXIST' failed. DEBUG1: Incoming command on sock [in consumer_thread_sessiond_poll() at consumer.c:2505] DEBUG1: Consumer received unexpected message size 0 (expects 4416) [in lttng_ustconsumer_recv_cmd() at ust-consumer.c:109] PERROR: sendmsg: Broken pipe [in lttcomm_send_unix_sock() at unix.c:204] DEBUG1: Communication interrupted on command socket [in consumer_thread_sessiond_poll() at consumer.c:2516] DEBUG1: consumer_thread_receive_fds exiting [in consumer_thread_sessiond_poll() at consumer.c:2526] DEBUG1: poll num_rdy : 1 [in consumer_thread_data_poll() at consumer.c:2251] DEBUG1: consumer_data_pipe wake up [in consumer_thread_data_poll() at consumer.c:2275] DEBUG1: Consumer delete flagged data stream [in validate_endpoint_status_data_stream() at consumer.c:1938] DEBUG1: polling thread exiting [in consumer_thread_data_poll() at consumer.c:2403] DEBUG1: Metadata event catched in thread [in consumer_thread_metadata_poll() at consumer.c:2030] DEBUG1: Metadata thread pipe hung up [in consumer_thread_metadata_poll() at consumer.c:2051] DEBUG1: Metadata poll thread exiting [in consumer_thread_metadata_poll() at consumer.c:2160] PERROR: sendmsg: Broken pipe [in lttcomm_send_unix_sock() at unix.c:204] DEBUG1: Consumer destroying it. Closing everything. [in lttng_consumer_destroy() at consumer.c:1196] Nov 14 12:33:43 SC-1 osafimmnd[6429]: Ccb 3872 COMMITTED (TraceC) 12487 12:33:42 11/14/2012 IN TraceC "DBG: Performing activate after relayD wait for session TestProfile1_1 12488 12:33:42 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12489 12:33:42 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x69D270 12490 12:33:42 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x693210 12491 12:33:42 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x69D170 12492 12:33:42 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x69E0B0 12493 12:33:42 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D270 12494 12:33:42 11/14/2012 IN TraceC "DBG: Invoking admin op 0x693210 12495 12:33:42 11/14/2012 IN TraceP_PL-3 "Activating 12496 12:33:42 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session activation requested, session name TestProfile1_1 with url net://192.168.0.1:59383:50886/./ 12497 12:33:42 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12498 12:33:42 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12499 12:33:42 11/14/2012 IN TraceP_PL-3 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12500 12:33:42 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -1 value: 1, return code: 1 , SA_AIS_OK 12501 12:33:42 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D170 12502 12:33:42 11/14/2012 IN TraceP_PL-4 "Activating 12503 12:33:42 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12504 12:33:42 11/14/2012 IN TraceP_SC-1 "Activating 12505 12:33:42 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12506 12:33:42 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69E0B0 12507 12:33:42 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session activation requested, session name TestProfile1_1 with url net://192.168.0.1:59383:50886/./ 12508 12:33:42 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12509 12:33:42 11/14/2012 NO TraceC "Session TestProfile1_1 activated. 12510 12:33:42 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG session activation requested, session name TestProfile1_1 with url net://192.168.0.1:59383:50886/./ 12511 12:33:42 11/14/2012 IN TraceP_SC-2 "Activating 12512 12:33:42 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12513 12:33:42 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12514 12:33:42 11/14/2012 IN TraceP_PL-4 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12515 12:33:42 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session activation requested, session name TestProfile1_1 with url net://192.168.0.1:59383:50886/./ 12516 12:33:42 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -2 value: 1, return code: 1 , SA_AIS_OK 12517 12:33:42 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12518 12:33:42 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12519 12:33:42 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12520 12:33:42 11/14/2012 IN TraceP_SC-2 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12521 12:33:42 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -4 value: 1, return code: 1 , SA_AIS_OK 12522 12:33:42 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12523 12:33:42 11/14/2012 IN TraceP_SC-1 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12524 12:33:42 11/14/2012 ER TraceP_SC-1 "Error: sessionD enable event operation failure for event com_ericsson_cba_trace_testapp_lowtraf ! error: No session daemon is available 12525 12:33:42 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -3 value: 20, return code: 1 , SA_AIS_OK Nov 14 12:33:43 SC-1 osafimmnd[6429]: Ccb 3873 COMMITTED (TraceC) Wed Nov 14 12:33:43 EST 2012 Allow session to run for 3 sec... 12526 12:33:43 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK 12527 12:33:43 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12528 12:33:43 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK 12529 12:33:43 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12530 12:33:43 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12531 12:33:43 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 20 error: 1 12532 12:33:43 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12533 12:33:43 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12534 12:33:43 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 20 12535 12:33:43 11/14/2012 ER TraceC "Error: TraceP "tracePId=SC-1,traceMId=1" could not activate Session "traceSessionRecordingId=TestProfile1_1,traceMId=1". Saf Error:SA_AIS_OK . Operation Result:SA_AIS_ERR_BAD_OPERATION. Nov 14 12:33:44 SC-1 logger: lttng-sessiond is started Wed Nov 14 12:33:46 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25505 24614 0 12:33 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:59383 -D tcp://192.168.0.1:50886 -o /home/trace/TestProfile1_1 root 25521 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:33:46 EST 2012 Printing Session 1 #################### Wed Nov 14 12:33:46 EST 2012 errors : tracePId=SC-1,traceMId=1: activation error Saf: SA_AIS_OK, Op: SA_AIS_ERR_BAD_OPERATION; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 11/14/12T12:33:39-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf traceProfileId : TestProfile1 -------------------------------- progress : "" Wed Nov 14 12:33:46 EST 2012 Deactivating Profile 1 #################### Wed Nov 14 12:33:46 EST 2012 Nov 14 12:33:47 SC-1 osafimmnd[6429]: Ccb 3876 COMMITTED (TraceC) 12536 12:33:46 11/14/2012 IN TraceC "DBG: Deactivating 12537 12:33:46 11/14/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProfile1_1,traceMId=1" from Registries 12538 12:33:46 11/14/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProfile1_1,traceMId=1" is valid 12539 12:33:46 11/14/2012 NO TraceC "Deactivating session TestProfile1_1. 12540 12:33:46 11/14/2012 IN TraceC "DBG: update Async for deactivate - first time 12541 12:33:46 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12542 12:33:46 11/14/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 12543 12:33:46 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x69DC70 12544 12:33:46 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x69DD30 12545 12:33:46 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x69DE30 12546 12:33:46 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x69DF80 12547 12:33:47 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:33:47 SC-1 osafimmnd[6429]: Ccb 3877 COMMITTED (TraceC) Nov 14 12:33:47 SC-1 osafimmnd[6429]: Ccb 3878 COMMITTED (TraceC) Wed Nov 14 12:33:47 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25521 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:33:47 EST 2012 Printing Profile 1 #################### Wed Nov 14 12:33:47 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 11/14/12T12:33:47-0500 progress.timeActionStarted : 11/14/12T12:33:46-0500 progress.timeOfLastStatusUpdate : 11/14/12T12:33:47-0500 Wed Nov 14 12:33:48 EST 2012 Deleting Session 1 #################### Wed Nov 14 12:33:48 EST 2012 Wed Nov 14 12:33:48 EST 2012 Deleting Profile 1 #################### Wed Nov 14 12:33:48 EST 2012 12548 12:33:47 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DC70 12549 12:33:47 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK 12550 12:33:47 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DD30 12551 12:33:47 11/14/2012 IN TraceP_PL-3 "Deactivating 12552 12:33:47 11/14/2012 IN TraceP_PL-3 "DBG: Session count 1 12553 12:33:47 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session stop tracing requested, session name TestProfile1_1 12554 12:33:47 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -5 value: 1, return code: 1 , SA_AIS_OK 12555 12:33:47 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DE30 12556 12:33:47 11/14/2012 IN TraceP_PL-4 "Deactivating 12557 12:33:47 11/14/2012 IN TraceP_PL-4 "DBG: Session count 1 12558 12:33:47 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session stop tracing requested, session name TestProfile1_1 12559 12:33:47 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12560 12:33:47 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -6 value: 1, return code: 1 , SA_AIS_OK 12561 12:33:47 11/14/2012 IN TraceP_SC-1 "Deactivating 12562 12:33:47 11/14/2012 IN TraceP_SC-1 "DBG: TraceP deactivate: Mo object traceSessionRecordingId=TestProfile1_1,traceMId=1 not found in registry. 12563 12:33:47 11/14/2012 IN TraceP_SC-1 "Cancelling 12564 12:33:47 11/14/2012 IN TraceP_SC-1 "DBG: TraceP Cancel: MO object traceSessionRecordingId=TestProfile1_1,traceMId=1 not found in registry ! 12565 12:33:47 11/14/2012 IN TraceP_SC-1 "DBG: Session count 0 12566 12:33:47 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -7 value: 1, return code: 1 , SA_AIS_OK 12567 12:33:47 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12568 12:33:47 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12569 12:33:47 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile1_1/PL-3 12570 12:33:47 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile1_1/PL-3. Stopped at /home/trace/TestProfile1_1. 12571 12:33:47 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12572 12:33:47 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12573 12:33:47 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile1_1/PL-4 12574 12:33:47 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile1_1/PL-4. Stopped at /home/trace/TestProfile1_1. 12575 12:33:47 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12576 12:33:47 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DF80 12577 12:33:47 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12578 12:33:47 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile1_1/SC-1 12579 12:33:47 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile1_1/SC-1. Stopped at /home/trace/TestProfile1_1. 12580 12:33:47 11/14/2012 IN TraceP_SC-2 "Deactivating 12581 12:33:47 11/14/2012 IN TraceP_SC-2 "DBG: Session count 1 12582 12:33:47 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session stop tracing requested, session name TestProfile1_1 12583 12:33:47 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -8 value: 1, return code: 1 , SA_AIS_OK 12584 12:33:47 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12585 12:33:47 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12586 12:33:47 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12587 12:33:47 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile1_1/SC-2 12588 12:33:47 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile1_1/SC-2. Stopped at /home/trace/TestProfile1_1. 12589 12:33:47 11/14/2012 IN TraceC "DBG: Terminating relayD for session TestProfile1_1 12590 12:33:47 11/14/2012 NO TraceC "Deactivate complete for session "TestProfile1_1" 12591 12:33:47 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile1_1 12592 12:33:47 11/14/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProfile1_1,traceMId=1 12593 12:33:47 11/14/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 12594 12:33:47 11/14/2012 IN TraceC "DBG: Received death of pid 25505 with ret 9. 12595 12:33:47 11/14/2012 IN TraceC "DBG: No handler for pid 25505. 12596 12:33:47 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile1,traceMId=1". Saf ErrorCode: SA_AIS_OK 12597 12:33:48 11/14/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProfile1_1,traceMId=1 12598 12:33:48 11/14/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProfile1_1,traceMId=1 12599 12:33:48 11/14/2012 IN TraceC "DBG: Try to async-remove directory "/home/trace//TestProfile1_1" 12600 12:33:48 11/14/2012 IN TraceC "DBG: remove called to delete file /home/trace//.tmp/TestProfile1_1 12601 12:33:48 11/14/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProfile1_1,traceMId=1 12602 12:33:48 11/14/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProfile1_1,traceMId=1 12603 12:33:48 11/14/2012 IN TraceC "DBG: Try to remove directory "/home/trace//.tmp/TestProfile1_1" 12604 12:33:48 11/14/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProfile1_1,traceMId=1 12605 12:33:48 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12606 12:33:48 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile1_1 has been deleted, moved, replaced or a delayed creation notification has been received in /home/trace 12607 12:33:48 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProfile1_1 12608 12:33:48 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProfile1_1 12609 12:33:48 11/14/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProfile1_1 is being deleted 12610 12:33:48 11/14/2012 IN TraceC "DBG: Remove Directory "/home/trace//.tmp/TestProfile1_1" Deleted! 12611 12:33:48 11/14/2012 IN TraceC "DBG: Completed deletion of "/home/trace//.tmp/TestProfile1_1" Nov 14 12:33:48 SC-1 osafimmnd[6429]: Ccb 3881 COMMITTED (TraceCLI) Wed Nov 14 12:33:48 EST 2012 Sleeping 2 sec before next run... ########################################################################### Wed Nov 14 12:33:50 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25521 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:33:50 EST 2012 Creating Profile 2 #################### Wed Nov 14 12:33:50 EST 2012 Nov 14 12:33:50 SC-1 osafimmnd[6429]: Ccb 3882 COMMITTED (TraceCLI) Wed Nov 14 12:33:50 EST 2012 Wait 1 sec... Nov 14 12:33:50 SC-1 osafimmnd[6429]: Ccb 3883 COMMITTED (TraceC) 12612 12:33:50 11/14/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 12613 12:33:50 11/14/2012 IN TraceC "DBG: newvalue size=27 12614 12:33:50 11/14/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProfile2 12615 12:33:50 11/14/2012 IN TraceC "DBG: ProfileAdmin::traceExpressionsValidator was called 12616 12:33:50 11/14/2012 IN TraceC "DBG: traceExpressions size=2 12617 12:33:50 11/14/2012 IN TraceC "DBG: About to create a temporary session /_TraceCValidator_reiPyv in path /tmp/_TraceCValidator_reiPyv 12618 12:33:50 11/14/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_reiPyv" 12619 12:33:50 11/14/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_reiPyv" Deleted! 12620 12:33:50 11/14/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProfile2,traceMId=1 Wed Nov 14 12:33:51 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25521 1 0 12:33 ? 00:00:00 lttng-sessiond -d root 25577 25521 0 12:33 ? 00:00:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error Wed Nov 14 12:33:51 EST 2012 Activating Profile 2 #################### Wed Nov 14 12:33:51 EST 2012 Nov 14 12:33:51 SC-1 osafimmnd[6429]: Ccb 3885 COMMITTED (TraceC) Nov 14 12:33:51 SC-1 osafimmnd[6429]: Ccb 3886 COMMITTED (TraceC) 12621 12:33:51 11/14/2012 IN TraceC "DBG: Activating 12622 12:33:51 11/14/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProfile2_1,traceMId=1 as a runtime object 12623 12:33:51 11/14/2012 NO TraceC "Activating Session TestProfile2_1. 12624 12:33:51 11/14/2012 IN TraceC "DBG: Directory /home/trace//TestProfile2_1 was created successfully! 12625 12:33:51 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12626 12:33:51 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12627 12:33:51 11/14/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 52777 12628 12:33:51 11/14/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 56915 12629 12:33:51 11/14/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 12630 12:33:51 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12631 12:33:51 11/14/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProfile2_1" pid: 25596 bound on 192.168.0.1:52777:56915 12632 12:33:51 11/14/2012 IN TraceC "DBG: Done Activating 12633 12:33:51 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile2_1 is being added to parent directory /home/trace 12634 12:33:51 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK 12635 12:33:51 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:33:54 SC-1 osafimmnd[6429]: Ccb 3887 COMMITTED (TraceC) Nov 14 12:33:54 SC-1 osafimmnd[6429]: Ccb 3888 COMMITTED (TraceC) 12636 12:33:54 11/14/2012 IN TraceC "DBG: Performing activate after relayD wait for session TestProfile2_1 12637 12:33:54 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12638 12:33:54 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x69DDA0 12639 12:33:54 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x69DEA0 12640 12:33:54 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x693E30 12641 12:33:54 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x6933D0 12642 12:33:54 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DDA0 12643 12:33:54 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DEA0 12644 12:33:54 11/14/2012 IN TraceP_PL-3 "Activating 12645 12:33:54 11/14/2012 IN TraceC "DBG: Invoking admin op 0x693E30 12646 12:33:54 11/14/2012 IN TraceP_PL-4 "Activating 12647 12:33:54 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session activation requested, session name TestProfile2_1 with url net://192.168.0.1:52777:56915/./ 12648 12:33:54 11/14/2012 IN TraceC "DBG: Invoking admin op 0x6933D0 12649 12:33:54 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12650 12:33:54 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12651 12:33:54 11/14/2012 IN TraceP_PL-3 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12652 12:33:54 11/14/2012 IN TraceP_SC-1 "Activating 12653 12:33:54 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session activation requested, session name TestProfile2_1 with url net://192.168.0.1:52777:56915/./ 12654 12:33:54 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12655 12:33:54 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12656 12:33:54 11/14/2012 IN TraceP_PL-4 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12657 12:33:54 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -9 value: 1, return code: 1 , SA_AIS_OK 12658 12:33:54 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG session activation requested, session name TestProfile2_1 with url net://192.168.0.1:52777:56915/./ 12659 12:33:54 11/14/2012 IN TraceP_SC-2 "Activating 12660 12:33:54 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -10 value: 1, return code: 1 , SA_AIS_OK 12661 12:33:54 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session activation requested, session name TestProfile2_1 with url net://192.168.0.1:52777:56915/./ 12662 12:33:54 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12663 12:33:54 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12664 12:33:54 11/14/2012 IN TraceP_SC-2 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12665 12:33:54 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12666 12:33:54 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -12 value: 1, return code: 1 , SA_AIS_OK 12667 12:33:54 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12668 12:33:54 11/14/2012 IN TraceP_SC-1 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12669 12:33:54 11/14/2012 ER TraceP_SC-1 "Error: sessionD enable event operation failure for event com_ericsson_cba_trace_testapp_lowtraf ! error: No session daemon is available 12670 12:33:54 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -11 value: 20, return code: 1 , SA_AIS_OK 12671 12:33:54 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12672 12:33:54 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK 12673 12:33:54 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12674 12:33:54 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12675 12:33:54 11/14/2012 NO TraceC "Session TestProfile2_1 activated. 12676 12:33:54 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12677 12:33:54 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12678 12:33:54 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 20 error: 1 12679 12:33:54 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12680 12:33:54 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12681 12:33:54 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 20 12682 12:33:54 11/14/2012 ER TraceC "Error: TraceP "tracePId=SC-1,traceMId=1" could not activate Session "traceSessionRecordingId=TestProfile2_1,traceMId=1". Saf Error:SA_AIS_OK . Operation Result:SA_AIS_ERR_BAD_OPERATION. 12683 12:33:54 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:33:55 EST 2012 Allow session to run for 3 sec... Nov 14 12:33:56 SC-1 dumpd: Removing dumps, limit reached (31 files, 2006M) Nov 14 12:33:56 SC-1 logger: lttng-sessiond is started Wed Nov 14 12:33:58 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25596 24614 0 12:33 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:52777 -D tcp://192.168.0.1:56915 -o /home/trace/TestProfile2_1 root 25612 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:33:58 EST 2012 Printing Session 2 #################### Wed Nov 14 12:33:58 EST 2012 errors : tracePId=SC-1,traceMId=1: activation error Saf: SA_AIS_OK, Op: SA_AIS_ERR_BAD_OPERATION; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 11/14/12T12:33:51-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf traceProfileId : TestProfile2 -------------------------------- progress : "" Wed Nov 14 12:33:58 EST 2012 Deactivating Profile 2 #################### Wed Nov 14 12:33:58 EST 2012 Nov 14 12:33:58 SC-1 osafimmnd[6429]: Ccb 3891 COMMITTED (TraceC) Nov 14 12:33:58 SC-1 osafimmnd[6429]: Ccb 3892 COMMITTED (TraceC) Nov 14 12:33:58 SC-1 osafimmnd[6429]: Ccb 3893 COMMITTED (TraceC) 12684 12:33:58 11/14/2012 IN TraceC "DBG: Deactivating 12685 12:33:58 11/14/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProfile2_1,traceMId=1" from Registries 12686 12:33:58 11/14/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProfile2_1,traceMId=1" is valid 12687 12:33:58 11/14/2012 NO TraceC "Deactivating session TestProfile2_1. 12688 12:33:58 11/14/2012 IN TraceC "DBG: update Async for deactivate - first time 12689 12:33:58 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12690 12:33:58 11/14/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 12691 12:33:58 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x694730 12692 12:33:58 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x69D070 12693 12:33:58 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x69D1A0 12694 12:33:58 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x69D2F0 12695 12:33:58 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK 12696 12:33:58 11/14/2012 IN TraceC "DBG: Invoking admin op 0x694730 12697 12:33:58 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK 12698 12:33:58 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D070 12699 12:33:58 11/14/2012 IN TraceP_PL-3 "Deactivating 12700 12:33:58 11/14/2012 IN TraceP_PL-3 "DBG: Session count 1 12701 12:33:58 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session stop tracing requested, session name TestProfile2_1 12702 12:33:58 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D1A0 12703 12:33:58 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -13 value: 1, return code: 1 , SA_AIS_OK 12704 12:33:58 11/14/2012 IN TraceP_PL-4 "Deactivating 12705 12:33:58 11/14/2012 IN TraceP_PL-4 "DBG: Session count 1 12706 12:33:58 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session stop tracing requested, session name TestProfile2_1 12707 12:33:58 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D2F0 12708 12:33:58 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -14 value: 1, return code: 1 , SA_AIS_OK 12709 12:33:58 11/14/2012 IN TraceP_SC-1 "Deactivating 12710 12:33:58 11/14/2012 IN TraceP_SC-1 "DBG: TraceP deactivate: Mo object traceSessionRecordingId=TestProfile2_1,traceMId=1 not found in registry. 12711 12:33:58 11/14/2012 IN TraceP_SC-1 "Cancelling 12712 12:33:58 11/14/2012 IN TraceP_SC-1 "DBG: TraceP Cancel: MO object traceSessionRecordingId=TestProfile2_1,traceMId=1 not found in registry ! 12713 12:33:58 11/14/2012 IN TraceP_SC-1 "DBG: Session count 0 12714 12:33:58 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -15 value: 1, return code: 1 , SA_AIS_OK 12715 12:33:58 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12716 12:33:58 11/14/2012 IN TraceP_SC-2 "Deactivating 12717 12:33:58 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12718 12:33:58 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12719 12:33:58 11/14/2012 IN TraceP_SC-2 "DBG: Session count 1 12720 12:33:58 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session stop tracing requested, session name TestProfile2_1 12721 12:33:58 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile2_1/PL-3 12722 12:33:58 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile2_1/PL-3. Stopped at /home/trace/TestProfile2_1. 12723 12:33:58 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -16 value: 1, return code: 1 , SA_AIS_OK 12724 12:33:58 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12725 12:33:58 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12726 12:33:58 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile2_1/PL-4 12727 12:33:58 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile2_1/PL-4. Stopped at /home/trace/TestProfile2_1. 12728 12:33:58 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12729 12:33:58 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12730 12:33:58 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile2_1/SC-1 12731 12:33:58 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile2_1/SC-1. Stopped at /home/trace/TestProfile2_1. 12732 12:33:58 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12733 12:33:58 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12734 12:33:58 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile2_1/SC-2 12735 12:33:58 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile2_1/SC-2. Stopped at /home/trace/TestProfile2_1. 12736 12:33:58 11/14/2012 IN TraceC "DBG: Terminating relayD for session TestProfile2_1 12737 12:33:58 11/14/2012 NO TraceC "Deactivate complete for session "TestProfile2_1" 12738 12:33:58 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile2_1 12739 12:33:58 11/14/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProfile2_1,traceMId=1 12740 12:33:58 11/14/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 12741 12:33:58 11/14/2012 IN TraceC "DBG: Received death of pid 25596 with ret 9. 12742 12:33:58 11/14/2012 IN TraceC "DBG: No handler for pid 25596. 12743 12:33:58 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile2,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:33:59 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25612 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:33:59 EST 2012 Printing Profile 2 #################### Wed Nov 14 12:33:59 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 11/14/12T12:33:58-0500 progress.timeActionStarted : 11/14/12T12:33:58-0500 progress.timeOfLastStatusUpdate : 11/14/12T12:33:58-0500 Wed Nov 14 12:33:59 EST 2012 Deleting Session 2 #################### Wed Nov 14 12:33:59 EST 2012 Wed Nov 14 12:33:59 EST 2012 Deleting Profile 2 #################### Wed Nov 14 12:33:59 EST 2012 Nov 14 12:33:59 SC-1 osafimmnd[6429]: Ccb 3896 COMMITTED (TraceCLI) Wed Nov 14 12:33:59 EST 2012 Sleeping 2 sec before next run... ########################################################################### 12744 12:33:59 11/14/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProfile2_1,traceMId=1 12745 12:33:59 11/14/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProfile2_1,traceMId=1 12746 12:33:59 11/14/2012 IN TraceC "DBG: Try to async-remove directory "/home/trace//TestProfile2_1" 12747 12:33:59 11/14/2012 IN TraceC "DBG: remove called to delete file /home/trace//.tmp/TestProfile2_1 12748 12:33:59 11/14/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProfile2_1,traceMId=1 12749 12:33:59 11/14/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProfile2_1,traceMId=1 12750 12:33:59 11/14/2012 IN TraceC "DBG: Try to remove directory "/home/trace//.tmp/TestProfile2_1" 12751 12:33:59 11/14/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProfile2_1,traceMId=1 12752 12:33:59 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12753 12:33:59 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile2_1 has been deleted, moved, replaced or a delayed creation notification has been received in /home/trace 12754 12:33:59 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProfile2_1 12755 12:33:59 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProfile2_1 12756 12:33:59 11/14/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProfile2_1 is being deleted 12757 12:33:59 11/14/2012 IN TraceC "DBG: Remove Directory "/home/trace//.tmp/TestProfile2_1" Deleted! 12758 12:33:59 11/14/2012 IN TraceC "DBG: Completed deletion of "/home/trace//.tmp/TestProfile2_1" Wed Nov 14 12:34:01 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25612 1 0 12:33 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:01 EST 2012 Creating Profile 3 #################### Wed Nov 14 12:34:01 EST 2012 Nov 14 12:34:01 SC-1 osafimmnd[6429]: Ccb 3897 COMMITTED (TraceCLI) Wed Nov 14 12:34:01 EST 2012 Wait 1 sec... Nov 14 12:34:01 SC-1 osafimmnd[6429]: Ccb 3898 COMMITTED (TraceC) 12759 12:34:01 11/14/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 12760 12:34:01 11/14/2012 IN TraceC "DBG: newvalue size=27 12761 12:34:01 11/14/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProfile3 12762 12:34:01 11/14/2012 IN TraceC "DBG: ProfileAdmin::traceExpressionsValidator was called 12763 12:34:01 11/14/2012 IN TraceC "DBG: traceExpressions size=2 12764 12:34:01 11/14/2012 IN TraceC "DBG: About to create a temporary session /_TraceCValidator_h6aKrx in path /tmp/_TraceCValidator_h6aKrx 12765 12:34:01 11/14/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_h6aKrx" 12766 12:34:01 11/14/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_h6aKrx" Deleted! 12767 12:34:01 11/14/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProfile3,traceMId=1 Wed Nov 14 12:34:02 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25612 1 0 12:33 ? 00:00:00 lttng-sessiond -d root 25667 25612 0 12:34 ? 00:00:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error Wed Nov 14 12:34:02 EST 2012 Activating Profile 3 #################### Wed Nov 14 12:34:02 EST 2012 Nov 14 12:34:02 SC-1 osafimmnd[6429]: Ccb 3900 COMMITTED (TraceC) Nov 14 12:34:02 SC-1 osafimmnd[6429]: Ccb 3901 COMMITTED (TraceC) 12768 12:34:02 11/14/2012 IN TraceC "DBG: Activating 12769 12:34:02 11/14/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProfile3_1,traceMId=1 as a runtime object 12770 12:34:02 11/14/2012 NO TraceC "Activating Session TestProfile3_1. 12771 12:34:02 11/14/2012 IN TraceC "DBG: Directory /home/trace//TestProfile3_1 was created successfully! 12772 12:34:02 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12773 12:34:02 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12774 12:34:02 11/14/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 57793 12775 12:34:02 11/14/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 58335 12776 12:34:02 11/14/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 12777 12:34:02 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12778 12:34:02 11/14/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProfile3_1" pid: 25684 bound on 192.168.0.1:57793:58335 12779 12:34:02 11/14/2012 IN TraceC "DBG: Done Activating 12780 12:34:02 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile3_1 is being added to parent directory /home/trace 12781 12:34:02 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK 12782 12:34:02 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:34:05 SC-1 osafimmnd[6429]: Ccb 3902 COMMITTED (TraceC) Nov 14 12:34:05 SC-1 osafimmnd[6429]: Ccb 3903 COMMITTED (TraceC) Nov 14 12:34:05 SC-1 logger: lttng-sessiond is started 12783 12:34:05 11/14/2012 IN TraceC "DBG: Performing activate after relayD wait for session TestProfile3_1 12784 12:34:05 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12785 12:34:05 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x69CF50 12786 12:34:05 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x69D0F0 12787 12:34:05 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x69D820 12788 12:34:05 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x69D200 12789 12:34:05 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69CF50 12790 12:34:05 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D0F0 12791 12:34:05 11/14/2012 IN TraceP_PL-3 "Activating 12792 12:34:05 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D820 12793 12:34:05 11/14/2012 IN TraceP_PL-4 "Activating 12794 12:34:05 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D200 12795 12:34:05 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session activation requested, session name TestProfile3_1 with url net://192.168.0.1:57793:58335/./ 12796 12:34:05 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12797 12:34:05 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12798 12:34:05 11/14/2012 IN TraceP_PL-3 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12799 12:34:05 11/14/2012 IN TraceP_SC-1 "Activating 12800 12:34:05 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session activation requested, session name TestProfile3_1 with url net://192.168.0.1:57793:58335/./ 12801 12:34:05 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12802 12:34:05 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12803 12:34:05 11/14/2012 IN TraceP_PL-4 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12804 12:34:05 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -17 value: 1, return code: 1 , SA_AIS_OK 12805 12:34:05 11/14/2012 IN TraceP_SC-2 "Activating 12806 12:34:05 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG session activation requested, session name TestProfile3_1 with url net://192.168.0.1:57793:58335/./ 12807 12:34:05 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -18 value: 1, return code: 1 , SA_AIS_OK 12808 12:34:05 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session activation requested, session name TestProfile3_1 with url net://192.168.0.1:57793:58335/./ 12809 12:34:05 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12810 12:34:05 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12811 12:34:05 11/14/2012 IN TraceP_SC-2 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12812 12:34:05 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12813 12:34:05 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -20 value: 1, return code: 1 , SA_AIS_OK 12814 12:34:05 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12815 12:34:05 11/14/2012 IN TraceP_SC-1 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12816 12:34:05 11/14/2012 ER TraceP_SC-1 "Error: sessionD enable event operation failure for event com_ericsson_cba_trace_testapp_lowtraf ! error: No session daemon is available 12817 12:34:05 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -19 value: 20, return code: 1 , SA_AIS_OK 12818 12:34:05 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12819 12:34:05 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK 12820 12:34:05 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12821 12:34:05 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12822 12:34:05 11/14/2012 NO TraceC "Session TestProfile3_1 activated. 12823 12:34:05 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12824 12:34:05 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12825 12:34:05 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 20 error: 1 12826 12:34:05 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12827 12:34:05 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12828 12:34:05 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 20 12829 12:34:05 11/14/2012 ER TraceC "Error: TraceP "tracePId=SC-1,traceMId=1" could not activate Session "traceSessionRecordingId=TestProfile3_1,traceMId=1". Saf Error:SA_AIS_OK . Operation Result:SA_AIS_ERR_BAD_OPERATION. 12830 12:34:05 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:34:06 EST 2012 Allow session to run for 3 sec... Wed Nov 14 12:34:09 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25684 24614 0 12:34 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:57793 -D tcp://192.168.0.1:58335 -o /home/trace/TestProfile3_1 root 25694 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:09 EST 2012 Printing Session 3 #################### Wed Nov 14 12:34:09 EST 2012 errors : tracePId=SC-1,traceMId=1: activation error Saf: SA_AIS_OK, Op: SA_AIS_ERR_BAD_OPERATION; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 11/14/12T12:34:02-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf traceProfileId : TestProfile3 -------------------------------- progress : "" Wed Nov 14 12:34:09 EST 2012 Deactivating Profile 3 #################### Wed Nov 14 12:34:09 EST 2012 Nov 14 12:34:09 SC-1 osafimmnd[6429]: Ccb 3906 COMMITTED (TraceC) Nov 14 12:34:09 SC-1 osafimmnd[6429]: Ccb 3907 COMMITTED (TraceC) Nov 14 12:34:09 SC-1 osafimmnd[6429]: Ccb 3908 COMMITTED (TraceC) 12831 12:34:09 11/14/2012 IN TraceC "DBG: Deactivating 12832 12:34:09 11/14/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProfile3_1,traceMId=1" from Registries 12833 12:34:09 11/14/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProfile3_1,traceMId=1" is valid 12834 12:34:09 11/14/2012 NO TraceC "Deactivating session TestProfile3_1. 12835 12:34:09 11/14/2012 IN TraceC "DBG: update Async for deactivate - first time 12836 12:34:09 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12837 12:34:09 11/14/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 12838 12:34:09 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x69DB80 12839 12:34:09 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x69DC40 12840 12:34:09 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x69DD70 12841 12:34:09 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x69DEC0 12842 12:34:09 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK 12843 12:34:09 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DB80 12844 12:34:09 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK 12845 12:34:09 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DC40 12846 12:34:09 11/14/2012 IN TraceP_PL-3 "Deactivating 12847 12:34:09 11/14/2012 IN TraceP_PL-3 "DBG: Session count 1 12848 12:34:09 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session stop tracing requested, session name TestProfile3_1 12849 12:34:09 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DD70 12850 12:34:09 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -21 value: 1, return code: 1 , SA_AIS_OK 12851 12:34:09 11/14/2012 IN TraceP_PL-4 "Deactivating 12852 12:34:09 11/14/2012 IN TraceP_PL-4 "DBG: Session count 1 12853 12:34:09 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session stop tracing requested, session name TestProfile3_1 12854 12:34:09 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DEC0 12855 12:34:09 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -22 value: 1, return code: 1 , SA_AIS_OK 12856 12:34:09 11/14/2012 IN TraceP_SC-1 "Deactivating 12857 12:34:09 11/14/2012 IN TraceP_SC-1 "DBG: TraceP deactivate: Mo object traceSessionRecordingId=TestProfile3_1,traceMId=1 not found in registry. 12858 12:34:09 11/14/2012 IN TraceP_SC-1 "Cancelling 12859 12:34:09 11/14/2012 IN TraceP_SC-1 "DBG: TraceP Cancel: MO object traceSessionRecordingId=TestProfile3_1,traceMId=1 not found in registry ! 12860 12:34:09 11/14/2012 IN TraceP_SC-1 "DBG: Session count 0 12861 12:34:09 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -23 value: 1, return code: 1 , SA_AIS_OK 12862 12:34:09 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12863 12:34:09 11/14/2012 IN TraceP_SC-2 "Deactivating 12864 12:34:09 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12865 12:34:09 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12866 12:34:09 11/14/2012 IN TraceP_SC-2 "DBG: Session count 1 12867 12:34:09 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session stop tracing requested, session name TestProfile3_1 12868 12:34:09 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile3_1/PL-3 12869 12:34:09 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile3_1/PL-3. Stopped at /home/trace/TestProfile3_1. 12870 12:34:09 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12871 12:34:09 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -24 value: 1, return code: 1 , SA_AIS_OK 12872 12:34:09 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12873 12:34:09 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile3_1/PL-4 12874 12:34:09 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile3_1/PL-4. Stopped at /home/trace/TestProfile3_1. 12875 12:34:09 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12876 12:34:09 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12877 12:34:09 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile3_1/SC-1 12878 12:34:09 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile3_1/SC-1. Stopped at /home/trace/TestProfile3_1. 12879 12:34:09 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12880 12:34:09 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 12881 12:34:09 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile3_1/SC-2 12882 12:34:09 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile3_1/SC-2. Stopped at /home/trace/TestProfile3_1. 12883 12:34:09 11/14/2012 IN TraceC "DBG: Terminating relayD for session TestProfile3_1 12884 12:34:09 11/14/2012 NO TraceC "Deactivate complete for session "TestProfile3_1" 12885 12:34:09 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile3_1 12886 12:34:09 11/14/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProfile3_1,traceMId=1 12887 12:34:09 11/14/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 12888 12:34:09 11/14/2012 IN TraceC "DBG: Received death of pid 25684 with ret 9. 12889 12:34:09 11/14/2012 IN TraceC "DBG: No handler for pid 25684. 12890 12:34:09 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile3,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:34:10 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25694 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:10 EST 2012 Printing Profile 3 #################### Wed Nov 14 12:34:10 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 11/14/12T12:34:09-0500 progress.timeActionStarted : 11/14/12T12:34:09-0500 progress.timeOfLastStatusUpdate : 11/14/12T12:34:09-0500 Wed Nov 14 12:34:10 EST 2012 Deleting Session 3 #################### Wed Nov 14 12:34:10 EST 2012 Wed Nov 14 12:34:10 EST 2012 Deleting Profile 3 #################### Wed Nov 14 12:34:10 EST 2012 Nov 14 12:34:10 SC-1 osafimmnd[6429]: Ccb 3911 COMMITTED (TraceCLI) Wed Nov 14 12:34:10 EST 2012 Sleeping 2 sec before next run... ########################################################################### 12891 12:34:10 11/14/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProfile3_1,traceMId=1 12892 12:34:10 11/14/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProfile3_1,traceMId=1 12893 12:34:10 11/14/2012 IN TraceC "DBG: Try to async-remove directory "/home/trace//TestProfile3_1" 12894 12:34:10 11/14/2012 IN TraceC "DBG: remove called to delete file /home/trace//.tmp/TestProfile3_1 12895 12:34:10 11/14/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProfile3_1,traceMId=1 12896 12:34:10 11/14/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProfile3_1,traceMId=1 12897 12:34:10 11/14/2012 IN TraceC "DBG: Try to remove directory "/home/trace//.tmp/TestProfile3_1" 12898 12:34:10 11/14/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProfile3_1,traceMId=1 12899 12:34:10 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12900 12:34:10 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile3_1 has been deleted, moved, replaced or a delayed creation notification has been received in /home/trace 12901 12:34:10 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProfile3_1 12902 12:34:10 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProfile3_1 12903 12:34:10 11/14/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProfile3_1 is being deleted 12904 12:34:10 11/14/2012 IN TraceC "DBG: Remove Directory "/home/trace//.tmp/TestProfile3_1" Deleted! 12905 12:34:10 11/14/2012 IN TraceC "DBG: Completed deletion of "/home/trace//.tmp/TestProfile3_1" Wed Nov 14 12:34:12 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25694 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:12 EST 2012 Creating Profile 4 #################### Wed Nov 14 12:34:12 EST 2012 Nov 14 12:34:12 SC-1 osafimmnd[6429]: Ccb 3912 COMMITTED (TraceCLI) Wed Nov 14 12:34:12 EST 2012 Wait 1 sec... Nov 14 12:34:12 SC-1 osafimmnd[6429]: Ccb 3913 COMMITTED (TraceC) 12906 12:34:12 11/14/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 12907 12:34:12 11/14/2012 IN TraceC "DBG: newvalue size=27 12908 12:34:12 11/14/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProfile4 12909 12:34:12 11/14/2012 IN TraceC "DBG: ProfileAdmin::traceExpressionsValidator was called 12910 12:34:12 11/14/2012 IN TraceC "DBG: traceExpressions size=2 12911 12:34:12 11/14/2012 IN TraceC "DBG: About to create a temporary session /_TraceCValidator_KxJih1 in path /tmp/_TraceCValidator_KxJih1 12912 12:34:12 11/14/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_KxJih1" 12913 12:34:12 11/14/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_KxJih1" Deleted! 12914 12:34:12 11/14/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProfile4,traceMId=1 Wed Nov 14 12:34:13 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25694 1 0 12:34 ? 00:00:00 lttng-sessiond -d root 25757 25694 0 12:34 ? 00:00:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error Wed Nov 14 12:34:13 EST 2012 Activating Profile 4 #################### Wed Nov 14 12:34:13 EST 2012 Nov 14 12:34:14 SC-1 osafimmnd[6429]: Ccb 3915 COMMITTED (TraceC) Nov 14 12:34:14 SC-1 osafimmnd[6429]: Ccb 3916 COMMITTED (TraceC) 12915 12:34:13 11/14/2012 IN TraceC "DBG: Activating 12916 12:34:14 11/14/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProfile4_1,traceMId=1 as a runtime object 12917 12:34:14 11/14/2012 NO TraceC "Activating Session TestProfile4_1. 12918 12:34:14 11/14/2012 IN TraceC "DBG: Directory /home/trace//TestProfile4_1 was created successfully! 12919 12:34:14 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12920 12:34:14 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12921 12:34:14 11/14/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 55386 12922 12:34:14 11/14/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 50492 12923 12:34:14 11/14/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 12924 12:34:14 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12925 12:34:14 11/14/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProfile4_1" pid: 25774 bound on 192.168.0.1:55386:50492 12926 12:34:14 11/14/2012 IN TraceC "DBG: Done Activating 12927 12:34:14 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile4_1 is being added to parent directory /home/trace 12928 12:34:14 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK 12929 12:34:14 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:34:16 SC-1 dumpd: Removing dumps, limit reached (31 files, 2006M) Nov 14 12:34:17 SC-1 osafimmnd[6429]: Ccb 3917 COMMITTED (TraceC) Nov 14 12:34:17 SC-1 osafimmnd[6429]: Ccb 3918 COMMITTED (TraceC) 12930 12:34:17 11/14/2012 IN TraceC "DBG: Performing activate after relayD wait for session TestProfile4_1 12931 12:34:17 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 12932 12:34:17 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x693690 12933 12:34:17 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x694AA0 12934 12:34:17 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x693200 12935 12:34:17 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x69D0F0 12936 12:34:17 11/14/2012 IN TraceC "DBG: Invoking admin op 0x693690 12937 12:34:17 11/14/2012 IN TraceC "DBG: Invoking admin op 0x694AA0 12938 12:34:17 11/14/2012 IN TraceP_PL-3 "Activating 12939 12:34:17 11/14/2012 IN TraceC "DBG: Invoking admin op 0x693200 12940 12:34:17 11/14/2012 IN TraceP_PL-4 "Activating 12941 12:34:17 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session activation requested, session name TestProfile4_1 with url net://192.168.0.1:55386:50492/./ 12942 12:34:17 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69D0F0 12943 12:34:17 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12944 12:34:17 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12945 12:34:17 11/14/2012 IN TraceP_PL-3 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12946 12:34:17 11/14/2012 IN TraceP_SC-1 "Activating 12947 12:34:17 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session activation requested, session name TestProfile4_1 with url net://192.168.0.1:55386:50492/./ 12948 12:34:17 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12949 12:34:17 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -25 value: 1, return code: 1 , SA_AIS_OK 12950 12:34:17 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12951 12:34:17 11/14/2012 IN TraceP_PL-4 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12952 12:34:17 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG session activation requested, session name TestProfile4_1 with url net://192.168.0.1:55386:50492/./ 12953 12:34:17 11/14/2012 IN TraceP_SC-2 "Activating 12954 12:34:17 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -26 value: 1, return code: 1 , SA_AIS_OK 12955 12:34:17 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session activation requested, session name TestProfile4_1 with url net://192.168.0.1:55386:50492/./ 12956 12:34:17 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12957 12:34:17 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12958 12:34:17 11/14/2012 IN TraceP_SC-2 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12959 12:34:17 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 12960 12:34:17 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -28 value: 1, return code: 1 , SA_AIS_OK 12961 12:34:17 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 12962 12:34:17 11/14/2012 IN TraceP_SC-1 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 12963 12:34:17 11/14/2012 ER TraceP_SC-1 "Error: sessionD enable event operation failure for event com_ericsson_cba_trace_testapp_lowtraf ! error: No session daemon is available 12964 12:34:17 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -27 value: 20, return code: 1 , SA_AIS_OK 12965 12:34:17 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK 12966 12:34:17 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12967 12:34:17 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12968 12:34:17 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12969 12:34:17 11/14/2012 NO TraceC "Session TestProfile4_1 activated. 12970 12:34:17 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 12971 12:34:17 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12972 12:34:17 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 20 error: 1 12973 12:34:17 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 12974 12:34:17 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 12975 12:34:17 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 20 12976 12:34:17 11/14/2012 ER TraceC "Error: TraceP "tracePId=SC-1,traceMId=1" could not activate Session "traceSessionRecordingId=TestProfile4_1,traceMId=1". Saf Error:SA_AIS_OK . Operation Result:SA_AIS_ERR_BAD_OPERATION. 12977 12:34:17 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:34:17 SC-1 TraceEa: ! sessiond faulted - Trying to Stop LTTNG Nov 14 12:34:17 SC-1 logger: lttng-sessiond is not running Wed Nov 14 12:34:18 EST 2012 Allow session to run for 3 sec... Nov 14 12:34:18 SC-1 logger: lttng-sessiond is started Wed Nov 14 12:34:21 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25774 24614 0 12:34 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:55386 -D tcp://192.168.0.1:50492 -o /home/trace/TestProfile4_1 root 25789 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:21 EST 2012 Printing Session 4 #################### Wed Nov 14 12:34:21 EST 2012 errors : tracePId=SC-1,traceMId=1: activation error Saf: SA_AIS_OK, Op: SA_AIS_ERR_BAD_OPERATION; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 11/14/12T12:34:14-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf traceProfileId : TestProfile4 -------------------------------- progress : "" Wed Nov 14 12:34:21 EST 2012 Deactivating Profile 4 #################### Wed Nov 14 12:34:21 EST 2012 Nov 14 12:34:21 SC-1 osafimmnd[6429]: Ccb 3921 COMMITTED (TraceC) Nov 14 12:34:21 SC-1 osafimmnd[6429]: Ccb 3922 COMMITTED (TraceC) 12978 12:34:21 11/14/2012 IN TraceC "DBG: Deactivating 12979 12:34:21 11/14/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProfile4_1,traceMId=1" from Registries 12980 12:34:21 11/14/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProfile4_1,traceMId=1" is valid 12981 12:34:21 11/14/2012 NO TraceC "Deactivating session TestProfile4_1. 12982 12:34:21 11/14/2012 IN TraceC "DBG: update Async for deactivate - first time 12983 12:34:21 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 12984 12:34:21 11/14/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 12985 12:34:21 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x69DD40 12986 12:34:21 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x6945D0 12987 12:34:21 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x694720 12988 12:34:21 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x694870 12989 12:34:21 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK 12990 12:34:21 11/14/2012 IN TraceC "DBG: Invoking admin op 0x69DD40 12991 12:34:21 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK 12992 12:34:21 11/14/2012 IN TraceC "DBG: Invoking admin op 0x6945D0 12993 12:34:21 11/14/2012 IN TraceP_PL-3 "Deactivating 12994 12:34:21 11/14/2012 IN TraceP_PL-3 "DBG: Session count 1 12995 12:34:21 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session stop tracing requested, session name TestProfile4_1 12996 12:34:21 11/14/2012 IN TraceC "DBG: Invoking admin op 0x694720 12997 12:34:21 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -29 value: 1, return code: 1 , SA_AIS_OK 12998 12:34:21 11/14/2012 IN TraceP_PL-4 "Deactivating 12999 12:34:21 11/14/2012 IN TraceP_PL-4 "DBG: Session count 1 13000 12:34:21 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session stop tracing requested, session name TestProfile4_1 13001 12:34:21 11/14/2012 IN TraceC "DBG: Invoking admin op 0x694870 13002 12:34:21 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -30 value: 1, return code: 1 , SA_AIS_OK 13003 12:34:21 11/14/2012 IN TraceP_SC-1 "Deactivating 13004 12:34:21 11/14/2012 IN TraceP_SC-1 "DBG: TraceP deactivate: Mo object traceSessionRecordingId=TestProfile4_1,traceMId=1 not found in registry. 13005 12:34:21 11/14/2012 IN TraceP_SC-1 "Cancelling 13006 12:34:21 11/14/2012 IN TraceP_SC-1 "DBG: TraceP Cancel: MO object traceSessionRecordingId=TestProfile4_1,traceMId=1 not found in registry ! 13007 12:34:21 11/14/2012 IN TraceP_SC-1 "DBG: Session count 0 13008 12:34:21 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -31 value: 1, return code: 1 , SA_AIS_OK 13009 12:34:21 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13010 12:34:21 11/14/2012 IN TraceP_SC-2 "Deactivating 13011 12:34:21 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13012 12:34:21 11/14/2012 IN TraceP_SC-2 "DBG: Session count 1 13013 12:34:21 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session stop tracing requested, session name TestProfile4_1 13014 12:34:21 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13015 12:34:21 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile4_1/PL-3 13016 12:34:21 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile4_1/PL-3. Stopped at /home/trace/TestProfile4_1. 13017 12:34:21 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -32 value: 1, return code: 1 , SA_AIS_OK 13018 12:34:21 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13019 12:34:21 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13020 12:34:21 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile4_1/PL-4 13021 12:34:21 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile4_1/PL-4. Stopped at /home/trace/TestProfile4_1. 13022 12:34:21 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13023 12:34:21 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13024 12:34:21 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile4_1/SC-1 13025 12:34:21 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile4_1/SC-1. Stopped at /home/trace/TestProfile4_1. 13026 12:34:21 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 13027 12:34:21 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13028 12:34:21 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile4_1/SC-2 13029 12:34:21 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile4_1/SC-2. Stopped at /home/trace/TestProfile4_1. 13030 12:34:21 11/14/2012 IN TraceC "DBG: Terminating relayD for session TestProfile4_1 Nov 14 12:34:21 SC-1 osafimmnd[6429]: Ccb 3923 COMMITTED (TraceC) Wed Nov 14 12:34:22 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25789 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:22 EST 2012 Printing Profile 4 #################### Wed Nov 14 12:34:22 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 11/14/12T12:34:21-0500 progress.timeActionStarted : 11/14/12T12:34:21-0500 progress.timeOfLastStatusUpdate : 11/14/12T12:34:21-0500 Wed Nov 14 12:34:22 EST 2012 Deleting Session 4 #################### Wed Nov 14 12:34:22 EST 2012 Wed Nov 14 12:34:22 EST 2012 Deleting Profile 4 #################### Wed Nov 14 12:34:22 EST 2012 13031 12:34:21 11/14/2012 NO TraceC "Deactivate complete for session "TestProfile4_1" 13032 12:34:21 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile4_1 13033 12:34:21 11/14/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProfile4_1,traceMId=1 13034 12:34:21 11/14/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 13035 12:34:21 11/14/2012 IN TraceC "DBG: Received death of pid 25774 with ret 9. 13036 12:34:21 11/14/2012 IN TraceC "DBG: No handler for pid 25774. 13037 12:34:21 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile4,traceMId=1". Saf ErrorCode: SA_AIS_OK 13038 12:34:22 11/14/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProfile4_1,traceMId=1 13039 12:34:22 11/14/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProfile4_1,traceMId=1 13040 12:34:22 11/14/2012 IN TraceC "DBG: Try to async-remove directory "/home/trace//TestProfile4_1" 13041 12:34:22 11/14/2012 IN TraceC "DBG: remove called to delete file /home/trace//.tmp/TestProfile4_1 13042 12:34:22 11/14/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProfile4_1,traceMId=1 13043 12:34:22 11/14/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProfile4_1,traceMId=1 13044 12:34:22 11/14/2012 IN TraceC "DBG: Try to remove directory "/home/trace//.tmp/TestProfile4_1" 13045 12:34:22 11/14/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProfile4_1,traceMId=1 13046 12:34:22 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 13047 12:34:22 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile4_1 has been deleted, moved, replaced or a delayed creation notification has been received in /home/trace 13048 12:34:22 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProfile4_1 13049 12:34:22 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProfile4_1 13050 12:34:22 11/14/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProfile4_1 is being deleted 13051 12:34:22 11/14/2012 IN TraceC "DBG: Remove Directory "/home/trace//.tmp/TestProfile4_1" Deleted! 13052 12:34:22 11/14/2012 IN TraceC "DBG: Completed deletion of "/home/trace//.tmp/TestProfile4_1" Nov 14 12:34:22 SC-1 osafimmnd[6429]: Ccb 3926 COMMITTED (TraceCLI) Wed Nov 14 12:34:22 EST 2012 Sleeping 2 sec before next run... ########################################################################### Wed Nov 14 12:34:24 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25789 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:24 EST 2012 Creating Profile 5 #################### Wed Nov 14 12:34:24 EST 2012 Nov 14 12:34:24 SC-1 osafimmnd[6429]: Ccb 3927 COMMITTED (TraceCLI) Wed Nov 14 12:34:24 EST 2012 Wait 1 sec... Nov 14 12:34:24 SC-1 osafimmnd[6429]: Ccb 3928 COMMITTED (TraceC) 13053 12:34:24 11/14/2012 IN TraceC "DBG: ProfileAdmin::profileIdValidator was called 13054 12:34:24 11/14/2012 IN TraceC "DBG: newvalue size=27 13055 12:34:24 11/14/2012 IN TraceC "DBG: newvalue=traceProfileId=TestProfile5 13056 12:34:24 11/14/2012 IN TraceC "DBG: ProfileAdmin::traceExpressionsValidator was called 13057 12:34:24 11/14/2012 IN TraceC "DBG: traceExpressions size=2 13058 12:34:24 11/14/2012 IN TraceC "DBG: About to create a temporary session /_TraceCValidator_4eEBdX in path /tmp/_TraceCValidator_4eEBdX 13059 12:34:24 11/14/2012 IN TraceC "DBG: Try to remove directory "/tmp/_TraceCValidator_4eEBdX" 13060 12:34:24 11/14/2012 IN TraceC "DBG: Remove Directory "/tmp/_TraceCValidator_4eEBdX" Deleted! 13061 12:34:24 11/14/2012 IN TraceC "DBG: Create: class: AsyncActionProgress, parentName: traceProfileId=TestProfile5,traceMId=1 Wed Nov 14 12:34:25 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25789 1 0 12:34 ? 00:00:00 lttng-sessiond -d root 25844 25789 0 12:34 ? 00:00:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error Wed Nov 14 12:34:25 EST 2012 Activating Profile 5 #################### Wed Nov 14 12:34:25 EST 2012 Nov 14 12:34:25 SC-1 osafimmnd[6429]: Ccb 3930 COMMITTED (TraceC) Nov 14 12:34:25 SC-1 osafimmnd[6429]: Ccb 3931 COMMITTED (TraceC) 13062 12:34:25 11/14/2012 IN TraceC "DBG: Activating 13063 12:34:25 11/14/2012 IN TraceC "DBG: creating session traceSessionRecordingId=TestProfile5_1,traceMId=1 as a runtime object 13064 12:34:25 11/14/2012 NO TraceC "Activating Session TestProfile5_1. 13065 12:34:25 11/14/2012 IN TraceC "DBG: Directory /home/trace//TestProfile5_1 was created successfully! 13066 12:34:25 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 13067 12:34:25 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 13068 12:34:25 11/14/2012 IN TraceC "DBG: RelayDManager::start() Control port selected: 56649 13069 12:34:25 11/14/2012 IN TraceC "DBG: RelayDManager::start() Data port selected: 51421 13070 12:34:25 11/14/2012 IN TraceC "DBG: RelayDManager::start() Pipe opened 13071 12:34:25 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 13072 12:34:25 11/14/2012 IN TraceC "DBG: RelayDManager::start() RelayD for "TestProfile5_1" pid: 25863 bound on 192.168.0.1:56649:51421 13073 12:34:25 11/14/2012 IN TraceC "DBG: Done Activating 13074 12:34:25 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile5_1 is being added to parent directory /home/trace 13075 12:34:25 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK 13076 12:34:25 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK Nov 14 12:34:28 SC-1 osafimmnd[6429]: Ccb 3932 COMMITTED (TraceC) Nov 14 12:34:28 SC-1 osafimmnd[6429]: Ccb 3933 COMMITTED (TraceC) 13077 12:34:28 11/14/2012 IN TraceC "DBG: Performing activate after relayD wait for session TestProfile5_1 13078 12:34:28 11/14/2012 IN TraceC "DBG: relayD ip address is found: 192.168.0.1 13079 12:34:28 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 2, addr: 0x7FEA48001910 13080 12:34:28 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 2, addr: 0x7FEA48001090 13081 12:34:28 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 2, addr: 0x7FEA48001250 13082 12:34:28 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 2, addr: 0x7FEA48000A50 13083 12:34:28 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48001910 13084 12:34:28 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48001090 13085 12:34:28 11/14/2012 IN TraceP_PL-3 "Activating 13086 12:34:28 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48001250 13087 12:34:28 11/14/2012 IN TraceP_PL-4 "Activating 13088 12:34:28 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session activation requested, session name TestProfile5_1 with url net://192.168.0.1:56649:51421/./ 13089 12:34:28 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48000A50 13090 12:34:28 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 13091 12:34:28 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 13092 12:34:28 11/14/2012 IN TraceP_PL-3 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 13093 12:34:28 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -33 value: 1, return code: 1 , SA_AIS_OK 13094 12:34:28 11/14/2012 IN TraceP_SC-1 "Activating 13095 12:34:28 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session activation requested, session name TestProfile5_1 with url net://192.168.0.1:56649:51421/./ 13096 12:34:28 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 13097 12:34:28 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 13098 12:34:28 11/14/2012 IN TraceP_PL-4 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 13099 12:34:28 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -34 value: 1, return code: 1 , SA_AIS_OK 13100 12:34:28 11/14/2012 IN TraceP_SC-2 "Activating 13101 12:34:28 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG session activation requested, session name TestProfile5_1 with url net://192.168.0.1:56649:51421/./ 13102 12:34:28 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session activation requested, session name TestProfile5_1 with url net://192.168.0.1:56649:51421/./ 13103 12:34:28 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 13104 12:34:28 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_hightraf:HundredPerSec 13105 12:34:28 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 13106 12:34:28 11/14/2012 IN TraceP_SC-2 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 13107 12:34:28 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -36 value: 1, return code: 1 , SA_AIS_OK 13108 12:34:28 11/14/2012 IN TraceP_SC-1 "DBG: LTTNG, enabling event, event name com_ericsson_cba_trace_testapp_lowtraf 13109 12:34:28 11/14/2012 IN TraceP_SC-1 "DBG: Dual event. enabling both com_ericsson_cba_trace_testapp_lowtraf:* and com_ericsson_cba_trace_testapp_lowtraf_* 13110 12:34:28 11/14/2012 ER TraceP_SC-1 "Error: sessionD enable event operation failure for event com_ericsson_cba_trace_testapp_lowtraf ! error: No session daemon is available 13111 12:34:28 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -35 value: 20, return code: 1 , SA_AIS_OK 13112 12:34:28 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13113 12:34:28 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK 13114 12:34:28 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13115 12:34:28 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 13116 12:34:28 11/14/2012 NO TraceC "Session TestProfile5_1 activated. 13117 12:34:28 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13118 12:34:28 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 13119 12:34:28 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 20 error: 1 13120 12:34:28 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 1 13121 12:34:28 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 13122 12:34:28 11/14/2012 IN TraceC "receiveTracePActivateResult() result: 1 op_result 20 13123 12:34:28 11/14/2012 ER TraceC "Error: TraceP "tracePId=SC-1,traceMId=1" could not activate Session "traceSessionRecordingId=TestProfile5_1,traceMId=1". Saf Error:SA_AIS_OK . Operation Result:SA_AIS_ERR_BAD_OPERATION. 13124 12:34:28 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:34:29 EST 2012 Allow session to run for 3 sec... Nov 14 12:34:30 SC-1 logger: lttng-sessiond is started Wed Nov 14 12:34:32 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25863 24614 0 12:34 ? 00:00:00 lttng-relayd -C tcp://192.168.0.1:56649 -D tcp://192.168.0.1:51421 -o /home/trace/TestProfile5_1 root 25879 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:32 EST 2012 Printing Session 5 #################### Wed Nov 14 12:34:32 EST 2012 errors : tracePId=SC-1,traceMId=1: activation error Saf: SA_AIS_OK, Op: SA_AIS_ERR_BAD_OPERATION; logLevelDefault : TRACE_DEBUG_SYSTEM(7) startTime : 11/14/12T12:34:25-0500 state : ACTIVE stopTime : "" traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf traceProfileId : TestProfile5 -------------------------------- progress : "" Wed Nov 14 12:34:32 EST 2012 Deactivating Profile 5 #################### Wed Nov 14 12:34:32 EST 2012 Nov 14 12:34:32 SC-1 osafimmnd[6429]: Ccb 3936 COMMITTED (TraceC) Nov 14 12:34:32 SC-1 osafimmnd[6429]: Ccb 3937 COMMITTED (TraceC) Nov 14 12:34:32 SC-1 osafimmnd[6429]: Ccb 3938 COMMITTED (TraceC) 13125 12:34:32 11/14/2012 IN TraceC "DBG: Deactivating 13126 12:34:32 11/14/2012 IN TraceC "DBG: Getting Session "traceSessionRecordingId=TestProfile5_1,traceMId=1" from Registries 13127 12:34:32 11/14/2012 IN TraceC "DBG: Checking if the current state of Session "traceSessionRecordingId=TestProfile5_1,traceMId=1" is valid 13128 12:34:32 11/14/2012 NO TraceC "Deactivating session TestProfile5_1. 13129 12:34:32 11/14/2012 IN TraceC "DBG: update Async for deactivate - first time 13130 12:34:32 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 13131 12:34:32 11/14/2012 IN TraceC "DBG: sending deactivate requests to all traceP's 13132 12:34:32 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-3,traceMId=1, op: 3, addr: 0x7FEA48001E80 13133 12:34:32 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=PL-4,traceMId=1, op: 3, addr: 0x7FEA48001F70 13134 12:34:32 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-1,traceMId=1, op: 3, addr: 0x7FEA48002030 13135 12:34:32 11/14/2012 IN TraceC "DBG: Creating admin op: dn tracePId=SC-2,traceMId=1, op: 3, addr: 0x7FEA48002160 13136 12:34:32 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK 13137 12:34:32 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48001E80 13138 12:34:32 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK 13139 12:34:32 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48001F70 13140 12:34:32 11/14/2012 IN TraceP_PL-3 "Deactivating 13141 12:34:32 11/14/2012 IN TraceP_PL-3 "DBG: Session count 1 13142 12:34:32 11/14/2012 IN TraceP_PL-3 "DBG: LTTNG session stop tracing requested, session name TestProfile5_1 13143 12:34:32 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48002030 13144 12:34:32 11/14/2012 IN TraceP_PL-3 "DBG: Return for invocation -37 value: 1, return code: 1 , SA_AIS_OK 13145 12:34:32 11/14/2012 IN TraceP_PL-4 "Deactivating 13146 12:34:32 11/14/2012 IN TraceC "DBG: Invoking admin op 0x7FEA48002160 13147 12:34:32 11/14/2012 IN TraceP_PL-4 "DBG: Session count 1 13148 12:34:32 11/14/2012 IN TraceP_PL-4 "DBG: LTTNG session stop tracing requested, session name TestProfile5_1 13149 12:34:32 11/14/2012 IN TraceP_PL-4 "DBG: Return for invocation -38 value: 1, return code: 1 , SA_AIS_OK 13150 12:34:32 11/14/2012 IN TraceP_SC-1 "Deactivating 13151 12:34:32 11/14/2012 IN TraceP_SC-1 "DBG: TraceP deactivate: Mo object traceSessionRecordingId=TestProfile5_1,traceMId=1 not found in registry. 13152 12:34:32 11/14/2012 IN TraceP_SC-1 "Cancelling 13153 12:34:32 11/14/2012 IN TraceP_SC-1 "DBG: TraceP Cancel: MO object traceSessionRecordingId=TestProfile5_1,traceMId=1 not found in registry ! 13154 12:34:32 11/14/2012 IN TraceP_SC-1 "DBG: Session count 0 13155 12:34:32 11/14/2012 IN TraceP_SC-1 "DBG: Return for invocation -39 value: 1, return code: 1 , SA_AIS_OK 13156 12:34:32 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13157 12:34:32 11/14/2012 IN TraceP_SC-2 "Deactivating 13158 12:34:32 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13159 12:34:32 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13160 12:34:32 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile5_1/PL-3 13161 12:34:32 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile5_1/PL-3. Stopped at /home/trace/TestProfile5_1. 13162 12:34:32 11/14/2012 IN TraceP_SC-2 "DBG: Session count 1 13163 12:34:32 11/14/2012 IN TraceP_SC-2 "DBG: LTTNG session stop tracing requested, session name TestProfile5_1 13164 12:34:32 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13165 12:34:32 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13166 12:34:32 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile5_1/PL-4 13167 12:34:32 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile5_1/PL-4. Stopped at /home/trace/TestProfile5_1. 13168 12:34:32 11/14/2012 IN TraceP_SC-2 "DBG: Return for invocation -40 value: 1, return code: 1 , SA_AIS_OK 13169 12:34:32 11/14/2012 IN TraceC "DBG: adminOperationInvokeAsyncCallback() operationReturnValue: 1 error: 1 13170 12:34:32 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13171 12:34:32 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile5_1/SC-1 13172 12:34:32 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile5_1/SC-1. Stopped at /home/trace/TestProfile5_1. 13173 12:34:32 11/14/2012 IN TraceC "DBG: saImmOmDispatch Succeeded 13174 12:34:32 11/14/2012 IN TraceC "receiveTracePDectivateResult() result: 1 op_result 1 13175 12:34:32 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile5_1/SC-2 13176 12:34:32 11/14/2012 IN TraceC "DBG: File Monitor seek: failed to find /home/trace/TestProfile5_1/SC-2. Stopped at /home/trace/TestProfile5_1. 13177 12:34:32 11/14/2012 IN TraceC "DBG: Terminating relayD for session TestProfile5_1 13178 12:34:32 11/14/2012 NO TraceC "Deactivate complete for session "TestProfile5_1" 13179 12:34:32 11/14/2012 IN TraceC "DBG: request to rescan directory TestProfile5_1 13180 12:34:32 11/14/2012 IN TraceC "DBG: deactivateSessionOnTracePsComplete calling TraceCHousekeeping::handleDelayedSessionWipes() for session traceSessionRecordingId=TestProfile5_1,traceMId=1 13181 12:34:32 11/14/2012 IN TraceC "DBG: SigChildHandler: second half handling. death count: 1 13182 12:34:32 11/14/2012 IN TraceC "DBG: Received death of pid 25863 with ret 9. 13183 12:34:32 11/14/2012 IN TraceC "DBG: No handler for pid 25863. 13184 12:34:32 11/14/2012 ER TraceC "Update AsyncActionProgress "id=1,traceProfileId=TestProfile5,traceMId=1". Saf ErrorCode: SA_AIS_OK Wed Nov 14 12:34:33 EST 2012 root 19533 1 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 19561 19533 0 11:30 ? 00:00:00 /opt/trace_ea/bin/TraceEa root 24614 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_c root 24622 1 0 12:25 ? 00:00:00 /opt/trace/sbin/trace_p root 25879 1 0 12:34 ? 00:00:00 lttng-sessiond -d Wed Nov 14 12:34:33 EST 2012 Printing Profile 5 #################### Wed Nov 14 12:34:33 EST 2012 consistentLogs : NOT_ENFORCED description : "" latestSessionId : 1 logLevelDefault : TRACE_DEBUG_SYSTEM(7) state : INACTIVE traceExpressions : com_ericsson_cba_trace_testapp_hightraf:HundredPerSec com_ericsson_cba_trace_testapp_lowtraf -------------------------------- progress.actionName : deactivateSession progress.additionalInfo : "" progress.info : Trace deactivation is complete progress.percentage : 100 progress.result : SUCCESS progress.resultInfo : Trace deactivateSession was completed successfully progress.state : FINISHED progress.timeActionCompleted : 11/14/12T12:34:32-0500 progress.timeActionStarted : 11/14/12T12:34:32-0500 progress.timeOfLastStatusUpdate : 11/14/12T12:34:32-0500 Wed Nov 14 12:34:33 EST 2012 Deleting Session 5 #################### Wed Nov 14 12:34:33 EST 2012 Wed Nov 14 12:34:33 EST 2012 Deleting Profile 5 #################### Wed Nov 14 12:34:33 EST 2012 Nov 14 12:34:33 SC-1 osafimmnd[6429]: Ccb 3941 COMMITTED (TraceCLI) Wed Nov 14 12:34:33 EST 2012 Sleeping 2 sec before next run... ########################################################################### 13185 12:34:33 11/14/2012 IN TraceC "DBG: Delete Session operation was requested for traceSessionRecordingId=TestProfile5_1,traceMId=1 13186 12:34:33 11/14/2012 IN TraceC "DBG: deleteSession Removing session Directory traceSessionRecordingId=TestProfile5_1,traceMId=1 13187 12:34:33 11/14/2012 IN TraceC "DBG: Try to async-remove directory "/home/trace//TestProfile5_1" 13188 12:34:33 11/14/2012 IN TraceC "DBG: remove called to delete file /home/trace//.tmp/TestProfile5_1 13189 12:34:33 11/14/2012 IN TraceC "DBG: deleteSession deleting ConvertAsyncActionProgress if it exists traceSessionRecordingId=TestProfile5_1,traceMId=1 13190 12:34:33 11/14/2012 IN TraceC "DBG: deleteSession deleting Session MO and removing it from Registry traceSessionRecordingId=TestProfile5_1,traceMId=1 13191 12:34:33 11/14/2012 IN TraceC "DBG: Try to remove directory "/home/trace//.tmp/TestProfile5_1" 13192 12:34:33 11/14/2012 IN TraceC "DBG: deleteSession deleting Session Object traceSessionRecordingId=TestProfile5_1,traceMId=1 13193 12:34:33 11/14/2012 IN TraceC "DBG: Return for invocation 1 value: 1, return code: 1 , SA_AIS_OK 13194 12:34:33 11/14/2012 IN TraceC "DBG: processMostQueuedEvents : directory TestProfile5_1 has been deleted, moved, replaced or a delayed creation notification has been received in /home/trace 13195 12:34:33 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping:directoryDeleteHandler is deleting TestProfile5_1 13196 12:34:33 11/14/2012 IN TraceC "DBG: TraceCHouseKeeping: directoryDeleteHandler is deleting TestProfile5_1 13197 12:34:33 11/14/2012 IN TraceC "DBG: directoryDeleteHandler : directory TestProfile5_1 is being deleted 13198 12:34:33 11/14/2012 IN TraceC "DBG: Remove Directory "/home/trace//.tmp/TestProfile5_1" Deleted! 13199 12:34:33 11/14/2012 IN TraceC "DBG: Completed deletion of "/home/trace//.tmp/TestProfile5_1" [4]+ Aborted lttng-sessiond -vvv --verbose-consumer --no-kernel > /cluster/temp/eusgarc/sc1_sessiond.log 2>&1 SC-1:/storage/no-backup/coremw/var/log/saflog # Nov 14 12:34:36 SC-1 dumpd: Removing dumps, limit reached (31 files, 2006M)