Project

General

Profile

Actions

Feature #484

closed

Provide alternative (more robust) form of liblttng-ust-cyg-profile function entry/exit instrumentation

Added by Paul Woegerer about 11 years ago. Updated almost 11 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
03/26/2013
Due date:
% Done:

0%

Estimated time:

Description

The current form of lttng_ust_cyg_profile:func_entry/exit has two disadvantages:
  • Lost events (even a few) make the function entry/exit instrumentation worthless.
  • Non-local returns (setjmp/longjmp, c++ exceptions, continuations) also make the instrumentation worthless.

Thus alternative to the lightweight variant another form of entry/exit instrumentation should be provided where
lttng_ust_cyg_profile:func_entry/exit record both arguments provided by the GCC hooks (this_fn and call_site).

This variant will be much more robust against lost (or missed) entry/exit events and non-local returns.

The following conversation on IRC motivated this feature request:

[13:39] <tgatterw> Compudj: Regarding function entry/exit instrumentation and the disadvantage not to pass the return address (when you attach with tracing to a running program) that pwoegere mentioned, there are more cases when the return address is important for us to have:
[13:39] <tgatterw> Compudj: When events are missed while tracing, the function enter/exit flow is broken. (I've seen this with more than one trace set.) With the return address in the events our tool currently still constructs the flow correctly after the missed events.
[13:39] <tgatterw> Compudj: Using setjmp/longjmp breaks function enter/exit relation and restarts at a previous point in the function call stack. Currently with the return address our tool correctly shows the functions flow after longjmp.
[13:40] <tgatterw> Compudj: And regarding the overhead, it's just one address more per event. 4 or 8 bytes.
[13:40] <Compudj> hi tgatterw,
[13:41] <Compudj> I understand that if there are events lost, it's harder to resync
[13:41] <Compudj> this is indeed a good point
[13:41] <Compudj> for setjmp/longjmp, even if you have the ret address, it's not a reliable way to handle this use-case
[13:42] <Compudj> e.g. if you have a() calls b() calls a() calls b() then longjmp to a()
[13:42] <Compudj> you don,t know which a() you're going back to
[13:43] <Compudj> for overhead, one address more per event when the major part of the overhead is function entry/exit events can make quite a difference
[13:44] <Compudj> I envision that a better way to handle setjmp/longjmp, exceptions, and continuations, would be to create a prefix tree that associates numeric ID to call stack states
[13:45] <Compudj> so we would save those numeric ID (that represent the state of the entire call stack) rather than just the top-of-stack ip address
[13:45] <tgatterw> Compudj: longjmp returns at the point where setjmp was called. So 'longjmp to a()' means a() has called setjmp before. I don't see why it would not be reliable. The function flow shown is correct, although hard to understand when you don't know that longjmp was involved.
[13:46] <Compudj> tgatterw: well, we don't instrument setjmp/longjmp at the moment
[13:46] <Compudj> so we don't have this info
[13:47] <Compudj> in the case where a calls b calls a calls b
[13:47] <Compudj> if we longjmp back into a
[13:47] <tgatterw> Compudj: Sure, but when you would add the return address to the exit event, the function flow would look correct even in that case.
[13:47] <Compudj> we don't know which a have called setjmp
[13:48] <Compudj> let's make this example a bit longer:
[13:48] <Compudj> a calls b calls a calls b calls a calls b
[13:48] <Compudj> if we longjmp back into e.g. a
[13:48] <Compudj> and return from a
[13:48] <Compudj> we don't know which "a" is was
[13:48] <Compudj> it was
[13:49] <tgatterw> a calls b : in events 'enter b, call_site=a'
[13:50] <Compudj> yep
[13:50] <tgatterw> b returns : in events 'exit b, call_site=a' (assuming that b was called from a)
[13:50] <Compudj> yep
[13:51] <Compudj> in my example, we'd calling a and b in turns, without returning
[13:51] <tgatterw> To construct the function enter/exit flow we don't need to know anything about previous events when we have both the this_function can call_site in each event.
[13:51] <Compudj> nesting 6 levels deep
[13:51] <tgatterw> I agree that the nesting level is not reconstructable here, but the function flow is correct.
[13:51] <Compudj> if you want to reconstruct the call stack, this is where you have an issue
[13:51] <tgatterw> Agreed!
[13:52] <tgatterw> For that case it would need instrumentation of setjmp/longjmp in any way.
[13:52] <Compudj> yep
[13:52] <Compudj> and you'd still have trouble with c++ exceptions
[13:52] <tgatterw> If they aren't using setjmp/longjmp ;)
[13:52] <Compudj> indeed, not sure about this one :)
[13:53] <Compudj> so are you saying we should also save the call site ?
[13:53] <Compudj> because it starts to make a big difference in overhead
[13:53] <Compudj> 1 pointer per entry/exit pair vs 4 pointers per pais
[13:53] <Compudj> pair
[13:54] <Compudj> that's about 4 times the overhead
[13:54] <Compudj> if we think of use-cases where users can afford to have very large memory buffers..
[13:54] <Compudj> where they don't discard events
[13:54] <tgatterw> Why 4 pointers? (I didn't look at the current implementation.) We just have one address for this_function and one address for call_site in our current instrumentation.
[13:54] <Compudj> it would be a shame to add overhead to their use-case just to handle the event discard situation
[13:55] <Compudj> func_entry: 2 pointers ?
[13:55] <Compudj> func_exit: 2 pointers.. ?
[13:55] <Compudj> total of 4 per entry/exit pair
[13:55] <tgatterw> I see, entry+exit together.
[13:56] <Compudj> we could possibly have two instrumentation sets
[13:56] <Compudj> each use-case seems to make sense and have its own value
[13:57] <tgatterw> Indeed. A lightweight version and one with full fct+call_site for special cases or debugging.
[13:58] <Compudj> lttng_ust_cyg_profile:func_entry_fast / lttng_ust_cyg_profile:func_exit_fast maybe ?
[13:59] <Compudj> and use func entry/exit for the full payload
[13:59] <tgatterw> Excellent idea.
[14:00] <Compudj> it could be interesting to make those two different .so maybe ?
[14:00] <Compudj> not sure
[14:01] <Compudj> given those are expected to be hit very, very often, it might be worth it
[14:01] <Compudj> so we could use the same event names
[14:02] <Compudj> ah, preferably not
[14:02] <Compudj> since the fields will differ
[14:02] <pwoegere> Compudj: its always good if a certain event type does not overload fields.
[14:03] <tgatterw> It would be possible to distinguish events by their payload, but a different event name would make it easier.
[14:03] <Compudj> different event names is required in fact
[14:04] <Compudj> e.g. if we have 2 processes with same event names but different fields, the second will be refused for per-UID buffers
[14:04] <tgatterw> Yes, that needs different event names.
[14:05] <tgatterw> And it would be interesting what would be needed in C++ case to correctly represent exceptions. If the (gcc) compiler has some instrumentation hooks for that too?
[14:05] <pwoegere> Compudj: it does if i remember correctly
[14:05] <tgatterw> (I think C++ exceptions are far more likely than setjmp/longjmp.)
[14:06] <Compudj> it would be good to put all that info into the feature request on the bug tracker
[14:06] <Compudj> will make it easier to remember it when we get to implementing it
[14:07] <pwoegere> Compudj, tgatterw: I will add another entry on bugs.lttng.org for this

Actions #1

Updated by Paul Woegerer almost 11 years ago

This is already implemented in LTTng-UST 2.2.0 RC1

Actions #2

Updated by Mathieu Desnoyers almost 11 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF