Project

General

Profile

Bug #1215

Failed to start lttng-sessiond process

Added by jinming liao about 1 month ago. Updated about 1 month ago.

Status:
Feedback
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
01/20/2020
Due date:
% Done:

0%

Estimated time:

Description

I manually compiled and installed the following packages:

userspace-rcu-0.11.0.tar.gz
lttng-ust-2.11.0.tar.gz
lttng-tools-2.11.0.tar.gz
babeltrace-1.5.7.tar.gz

Version of my environment:

[root@localhost ~]# uname -a
Linux localhost.localdomain 3.10.0-514.44.5.10.h165.x86_64 #1 SMP Tue Feb 26 18:43:36 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

When I start the lttng-sessiond process, I get the following error and the startup fails:

[root@localhost /]# lttng-sessiond --no-kernel --daemonize
PERROR - 21:59:39.270365940 [13146/13146]: remove lock file: No such file or directory (in sessiond_cleanup_lock_file() at main.c:917)

The details are as follows:

[root@localhost /]# lttng-sessiond --no-kernel --daemonize -vvv
DEBUG1 - 21:59:44.528196981 [14732/14732]: [sessiond configuration]
DEBUG1 - 21:59:44.528243054 [14732/14732]:     verbose:                       3
DEBUG1 - 21:59:44.528247187 [14732/14732]:     verbose consumer:              0
DEBUG1 - 21:59:44.528250600 [14732/14732]:     quiet mode:                    False
DEBUG1 - 21:59:44.528253731 [14732/14732]:     agent_tcp_port:                [5345, 5354]
DEBUG1 - 21:59:44.528257251 [14732/14732]:     application socket timeout:    5
DEBUG1 - 21:59:44.528260396 [14732/14732]:     no-kernel:                     True
DEBUG1 - 21:59:44.528263364 [14732/14732]:     background:                    False
DEBUG1 - 21:59:44.528266333 [14732/14732]:     daemonize:                     True
DEBUG1 - 21:59:44.528269469 [14732/14732]:     signal parent on start:        False
DEBUG1 - 21:59:44.528272322 [14732/14732]:     tracing group name:            tracing
DEBUG1 - 21:59:44.528275269 [14732/14732]:     kmod_probe_list:               None
DEBUG1 - 21:59:44.528278401 [14732/14732]:     kmod_extra_probe_list:         None
DEBUG1 - 21:59:44.528281599 [14732/14732]:     rundir:                        /var/run/lttng
DEBUG1 - 21:59:44.528284756 [14732/14732]:     application socket path:       /var/run/lttng/lttng-ust-sock-8
DEBUG1 - 21:59:44.528289975 [14732/14732]:     client socket path:            /var/run/lttng/client-lttng-sessiond
DEBUG1 - 21:59:44.528293229 [14732/14732]:     wait shm path:                 /lttng-ust-wait-8
DEBUG1 - 21:59:44.528296428 [14732/14732]:     health socket path:            /var/run/lttng/sessiond-health
DEBUG1 - 21:59:44.528299639 [14732/14732]:     LTTNG_UST_CLOCK_PLUGIN:        None
DEBUG1 - 21:59:44.528302575 [14732/14732]:     pid file path:                 /var/run/lttng/lttng-sessiond.pid
DEBUG1 - 21:59:44.528305681 [14732/14732]:     lock file path:                /var/run/lttng/lttng-sessiond.lck
DEBUG1 - 21:59:44.528308690 [14732/14732]:     session load path:             None
DEBUG1 - 21:59:44.528311621 [14732/14732]:     agent port file path:          /var/run/lttng/agent.port
DEBUG1 - 21:59:44.528320498 [14732/14732]:     consumerd32 path:              /var/run/lttng/ustconsumerd32
DEBUG1 - 21:59:44.528323678 [14732/14732]:     consumerd32 bin path:          Unknown
DEBUG1 - 21:59:44.528326739 [14732/14732]:     consumerd32 lib dir:           Unknown
DEBUG1 - 21:59:44.528329788 [14732/14732]:     consumerd32 err unix sock path:/var/run/lttng/ustconsumerd32/error
DEBUG1 - 21:59:44.528332689 [14732/14732]:     consumerd32 cmd unix sock path:/var/run/lttng/ustconsumerd32/command
DEBUG1 - 21:59:44.528335552 [14732/14732]:     consumerd64 path:              /var/run/lttng/ustconsumerd64
DEBUG1 - 21:59:44.528338710 [14732/14732]:     consumerd64 bin path:          /usr/lib64/lttng/libexec/lttng-consumerd
DEBUG1 - 21:59:44.528341818 [14732/14732]:     consumerd64 lib dir:           /usr/lib64
DEBUG1 - 21:59:44.528344781 [14732/14732]:     consumerd64 err unix sock path:/var/run/lttng/ustconsumerd64/error
DEBUG1 - 21:59:44.528347947 [14732/14732]:     consumerd64 cmd unix sock path:/var/run/lttng/ustconsumerd64/command
DEBUG1 - 21:59:44.528351043 [14732/14732]:     kconsumerd path:               /var/run/lttng/kconsumerd
DEBUG1 - 21:59:44.528354134 [14732/14732]:     kconsumerd err unix sock path: /var/run/lttng/kconsumerd/error
DEBUG1 - 21:59:44.528357114 [14732/14732]:     kconsumerd cmd unix sock path: /var/run/lttng/kconsumerd/command
DEBUG1 - 21:59:44.528364853 [14732/14732]: Starting lttng-sessiond {aef25127-dc4c-4a92-9c2d-f4c7e719adc8} (in sessiond_uuid_log() at main.c:1288)
DEBUG3 - 21:59:44.528370373 [14732/14732]: Creating LTTng run directory: /var/run/lttng (in create_lttng_rundir() at main.c:1001)
PERROR - 21:59:45.528724959 [14732/14732]: remove lock file: No such file or directory (in sessiond_cleanup_lock_file() at main.c:917)
DEBUG1 - 21:59:45.528753860 [14732/14732]: Cleaning up options (in sessiond_cleanup_options() at main.c:354)
DEBUG1 - 21:59:45.528764827 [14732/14732]: Destroying run_as worker (in run_as_destroy_worker_no_lock() at runas.c:1280)

I tracked the start of lttng-sessiond and found that CMM_LOAD_SHARED failed and the child process exited:

src/common/daemonize.c:lttng_daemonize:102

while (!CMM_LOAD_SHARED(*completion_flag)) {
    int status;
    pid_t ret;

    /*
     * Check if child exists without blocking. If
     * so, we have to stop this parent process and
     * return an error.
     */
    ret = waitpid(pid, &status, WNOHANG);
    if (ret < 0 || (ret != 0 && WIFEXITED(status))) {
        /* The child exited somehow or was not valid. */
        goto error;
    }
    sleep(1);
}

What is CMM_LOAD_SHARED? Is it not supported by my system?

#1

Updated by Jonathan Rajotte Julien about 1 month ago

  • Status changed from New to Feedback

Hi,

Let's focus a bit on the initial PERROR:

Note that with the daemonize option, we fork and wait for the child to inform the parent to exit (via the completion flag). We also close all output fds.

Does starting without the --daemoize option works?

As for the CMM_LOAD_SHARED: https://lwn.net/Articles/573435/
Given that you seem to be running x86_64 I doubt this is not supported.

Can you give us more information on the platform you are using? Is it in a container? Is it baremetal? Is it a vm? What distro?

Cheers

#2

Updated by jinming liao about 1 month ago

Is a physical machine, and after testing, it is ok in the virtual machine and container.

Also available in: Atom PDF