Project

General

Profile

Actions

Bug #1283

closed

Problem with lttng load on 2.12

Added by Filip Pudak over 3 years ago. Updated about 3 years ago.

Status:
Won't fix
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
09/17/2020
Due date:
% Done:

0%

Estimated time:

Description

Hi, I just upgraded from 2.10 -> 2.12 and have encountered this issue while running my usual script.
Console output :

---------------


+ lttng_msg=-q
+ session_cfg_path=/tmp/lttng_session_cfg
+ lttng -q save --force --output-path=/tmp/lttng_session_cfg session_trace
+ exit_status=0
+ '[' 0 -eq 0 ']'
+ lttng -q stop session_trace
+ lttng -q destroy session_trace
+ sleep 1
+ rm -rf '/var/log/lttng/shm/save/*' /var/log/lttng/shm/current
+ df /pramfs -h
Filesystem Size Used Avail Use% Mounted on
/dev/pmem0 488M 2.7M 456M 1% /pramfs
+ lttng -vvv load --input-path=/tmp/lttng_session_cfg
DEBUG3 - 13:51:54.740354470 [3561/3561]: URI string: file:///var/volatile/tmp/lttng_session_cfg (in uri_parse() at lttng-tools/2.12.0-r0/git/src/common/uri.c:298)
DEBUG3 - 13:51:54.740476330 [3561/3561]: URI file destination: /var/volatile/tmp/lttng_session_cfg (in uri_parse() at lttng-tools/2.12.0-r0/git/src/common/uri.c:335)
DEBUG3 - 13:51:54.740498090 [3561/3561]: URI dtype: 3, proto: 0, host: , subdir: , ctrl: 0, data: 0 (in uri_parse() at lttng-tools/2.12.0-r0/git/src/common/uri.c:512)
DEBUG1 - 13:51:54.745829510 [3561/3561]: LSM cmd type : 49 (in send_session_msg() at lttng-tools/2.12.0-r0/git/src/lib/lttng-ctl/lttng-ctl.c:143)
DEBUG1 - 13:51:54.746218790 [3561/3561]: LSM cmd type : 25 (in send_session_msg() at lttng-tools/2.12.0-r0/git/src/lib/lttng-ctl/lttng-ctl.c:143)
DEBUG1 - 13:51:54.746497020 [3561/3561]: LSM cmd type : 40 (in send_session_msg() at lttng-tools/2.12.0-r0/git/src/lib/lttng-ctl/lttng-ctl.c:143)
*Error: Failed to load session session_trace: Not enough memory*
DEBUG1 - 13:51:54.746995900 [3561/3561]: LSM cmd type : 17 (in send_session_msg() at lttng-tools/2.12.0-r0/git/src/lib/lttng-ctl/lttng-ctl.c:143)
DEBUG1 - 13:51:54.747147820 [3561/3561]: LSM cmd type : 24 (in send_session_msg() at lttng-tools/2.12.0-r0/git/src/lib/lttng-ctl/lttng-ctl.c:143)
DEBUG1 - 13:51:54.747353630 [3561/3561]: *epoll set max size is 1024860*  (in compat_epoll_set_max_size() at lttng-tools/2.12.0-r0/git/src/common/compat/compat-epoll.c:346)
*Error: Not enough memory*
DEBUG1 - 13:51:54.748448160 [3561/3561]: Clean exit (in clean_exit() at lttng-tools/2.12.0-r0/git/src/bin/lttng/lttng.c:152)


--------------
Has anyone encountered a similar issue? I know there is lttng clear in 2.12, but would still like to understand why the OOM crash is there...

/Filip

Actions #1

Updated by Jonathan Rajotte Julien over 3 years ago

  • Description updated (diff)

Hi,

Please share the session configuration? (xml file)

Please provide the output from the following command?

df -h
free -h

Also please provide the verbose output from lttng-sessiond:

lttng-sessiond -vvv --verbose-consumer

I do not think lttng clear have anything to do with the current problem.

Is the problem always reproducible? Are you able to create a similar configuration manually after the load fails?

Thanks

Actions #2

Updated by Jonathan Rajotte Julien over 3 years ago

  • Status changed from New to Feedback
Actions #3

Updated by Filip Pudak over 3 years ago

Hi Jonathan,

I worked around this by using lttng_clear in the end.
The issue was reproducible.
We can close this case.

Cheers,
Filip

Actions #4

Updated by Jonathan Rajotte Julien over 3 years ago

Hi Filip,

I understand you found a work around here. Keep in mind that here the clear option probably work only because we do not allocate new buffers for the session, we simply clear them.

Still it seems that there is a problem/quirk regarding the lifetime of the allocated memory for the buffers. Which is probably why on the "load" you are seeing allocation failure.

It would really help us to get the output from the following command on error:

lttng-sessiond -vvv --verbose-consumer

Cheers

Actions #5

Updated by Jonathan Rajotte Julien about 3 years ago

  • Status changed from Feedback to Won't fix
Actions

Also available in: Atom PDF