Project

General

Profile

Actions

Bug #1334

open

Python component class memory leak

Added by Simon Marchi almost 3 years ago. Updated almost 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Start date:
11/29/2021
Due date:
% Done:

0%

Estimated time:

Description

I saw this memory leak when working on a Python sink:

/tmp/babeltrace/bin/babeltrace2 --plugin-path=$PWD net://localhost/host/simark/py_logging  -c sink.hello.Hello
11-29 21:49:09.490 566263 566263 W BT2-PY log_exception_and_maybe_append_cause@native_bt_log_and_append_error.h:204 [sink.hello.Hello] Traceback (most recent call last):
  File "/home/simark/src/PyConCanada2017/bt_plugin_foo.py", line 10, in _user_consume
    raise ValueError("hi")
ValueError: hi
11-29 21:49:09.490 566263 566263 W LIB/GRAPH consume_graph_sink@graph.c:462 Component's "consume" method failed: status=ERROR, comp-addr=0x60c000066ac0, comp-name="sink.hello.Hello", comp-log-level=WARNING, comp-class-type=SINK, comp-class-name="Hello", comp-class-partial-descr="", comp-class-is-frozen=1, comp-input-port-count=1, comp-output-port-count=0
11-29 21:49:09.490 566263 566263 E CLI cmd_run@babeltrace2.c:2537 Graph failed to complete successfully

ERROR:    [Babeltrace CLI] (/home/simark/src/babeltrace/src/cli/babeltrace2.c:2537)
  Graph failed to complete successfully
CAUSED BY [libbabeltrace2] (/home/simark/src/babeltrace/src/lib/graph/graph.c:462)
  Component's "consume" method failed: status=ERROR, comp-addr=0x60c000066ac0, comp-name="sink.hello.Hello", comp-log-level=WARNING, comp-class-type=SINK,
  comp-class-name="Hello", comp-class-partial-descr="", comp-class-is-frozen=1, comp-input-port-count=1, comp-output-port-count=0
CAUSED BY [sink.hello.Hello: 'sink.hello.Hello'] (/home/simark/src/babeltrace/src/bindings/python/bt2/bt2/native_bt_log_and_append_error.h:102)
  Traceback (most recent call last):
    File "/home/simark/src/PyConCanada2017/bt_plugin_foo.py", line 10, in _user_consume
      raise ValueError("hi")
  ValueError: hi

=================================================================
==566263==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 184 byte(s) in 1 object(s) allocated from:
    #0 0x7f10290c2459 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f1028d48121 in g_malloc0 (/usr/lib/libglib-2.0.so.0+0x5d121)
    #2 0x7f1024605c08 in bt_bt2_component_class_sink_create /home/simark/src/babeltrace/src/bindings/python/bt2/bt2/native_bt_component_class.i.h:1485
    #3 0x7f1024622599 in _wrap_bt2_component_class_sink_create bt2/native_bt.c:7300
    #4 0x7f1025090c37  (/usr/lib/libpython3.9.so.1.0+0x148c37)

SUMMARY: AddressSanitizer: 184 byte(s) leaked in 1 allocation(s).

The file containing the component class:

import bt2

@bt2.plugin_component_class
class Hello(bt2._UserSinkComponent):
    def __init__(self, config, params, obj):
        self._port = self._add_input_port("port")

    def _user_consume(self):
        raise ValueError("hi")

bt2.register_plugin(__name__, "hello")

Actions #1

Updated by Simon Marchi almost 3 years ago

Actually, it does not look related to the fact that an exception is raised in _user_consume. I get the same leak when the graph executes successfully. With this code:

import bt2

@bt2.plugin_component_class
class Hello(bt2._UserSinkComponent):
    def __init__(self, config, params, obj):
        self._port = self._add_input_port("port")

    def _user_graph_is_configured(self):
        self._msg_iter = self._create_message_iterator(self._port)

    def _user_consume(self):
        next(self._msg_iter)

bt2.register_plugin(__name__, "hello")
$ /tmp/babeltrace/bin/babeltrace2 --plugin-path=$PWD ~/lttng-traces/simark  -c sink.hello.Hello

=================================================================
==569406==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 184 byte(s) in 1 object(s) allocated from:
    #0 0x7fdc5ef8d459 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7fdc5ec13121 in g_malloc0 (/usr/lib/libglib-2.0.so.0+0x5d121)
    #2 0x7fdc5a505c08 in bt_bt2_component_class_sink_create /home/simark/src/babeltrace/src/bindings/python/bt2/bt2/native_bt_component_class.i.h:1485
    #3 0x7fdc5a522599 in _wrap_bt2_component_class_sink_create bt2/native_bt.c:7300
    #4 0x7fdc5af90c37  (/usr/lib/libpython3.9.so.1.0+0x148c37)

SUMMARY: AddressSanitizer: 184 byte(s) leaked in 1 allocation(s).

Note that I'm using the PYTHONMALLOC=malloc env var, so normally everything should be cleanly freed.

Actions #2

Updated by Simon Marchi almost 3 years ago

  • Subject changed from Python component class memory leak when exception is raised from _user_consume to Python component class memory leak
Actions

Also available in: Atom PDF