LTTng » LTTng-tools

I'm not sure what you mean by "it should be noted that an event list (for instance) can be optimized against lookup by ensuring the most-frequently requested events appear at the head of the list."

Do you mean improving the tracers' performance by ordering the events in a particular way?

Yes. Suppose 90% of my events are of a certain type and are assigned to a certain channel while the remaining 10% are spread across a dozen or so other channels. If the majority event's channel is the first in the channel list, every lookup for that channel will terminate faster than if it is at the end of the list. That's just a few instruction cycles, but every little bit helps sometimes.

If the channel were looked up every time the event occurs, it would mean large savings, but I don't think that's how LTTng works (I suspect the session daemon looks up the event type once at session start and sets up the tracepoint providers so they have the channel pointer in persistent memory). It may be more important in user-space, where events can register and unregister repeatedly during a session.

The annoyance of having the channel list invert with every session load/save cycle would also cause problems during trace analysis, as the babeltrace API recovers channel IDs and not channel names: a user creating a collection of several sessions, some of which are using an inverted channel list with respect to the other sessions, would be puzzled by the changing channel IDs (TRACE_PACKET_HEADER stream_id) of his events.

Daniel U. Thibault wrote:

Yes. Suppose 90% of my events are of a certain type and are assigned to a certain channel while the remaining 10% are spread across a dozen or so other channels. If the majority event's channel is the first in the channel list, every lookup for that channel will terminate faster than if it is at the end of the list. That's just a few instruction cycles, but every little bit helps sometimes.

I will need to see the difference in a benchmark to justify changing this on the grounds of performance concerns. I'm not convinced that it makes a noticeable difference in practice, but I'm open to being proven wrong.

If the channel were looked up every time the event occurs, it would mean large savings, but I don't think that's how LTTng works (I suspect the session daemon looks up the event type once at session start and sets up the tracepoint providers so they have the channel pointer in persistent memory). It may be more important in user-space, where events can register and unregister repeatedly during a session.

There are no such look-ups happening at run-time, as you pointed out.

The annoyance of having the channel list invert with every session load/save cycle would also cause problems during trace analysis, as the babeltrace API recovers channel IDs and not channel names: a user creating a collection of several sessions, some of which are using an inverted channel list with respect to the other sessions, would be puzzled by the changing channel IDs (TRACE_PACKET_HEADER stream_id) of his events.

There should indeed be a way to recover a human-readable channel name (or stream name, in CTF parlance). However, the tracer offers no guarantee that channel IDs will be preserved from one session to the next. The fact that this id is generated in such a "predictable" way is an implementation detail that is not guaranteed to remain unchanged in the future.

Jérémie Galarneau wrote:

Daniel U. Thibault wrote:

The annoyance of having the channel list invert with every session load/save cycle would also cause problems during trace analysis, as the babeltrace API recovers channel IDs and not channel names: a user creating a collection of several sessions, some of which are using an inverted channel list with respect to the other sessions, would be puzzled by the changing channel IDs (TRACE_PACKET_HEADER stream_id) of his events.

There should indeed be a way to recover a human-readable channel name (or stream name, in CTF parlance). However, the tracer offers no guarantee that channel IDs will be preserved from one session to the next. The fact that this id is generated in such a "predictable" way is an implementation detail that is not guaranteed to remain unchanged in the future.

Until the babeltrace API offers a way of recovering the channel names, the users have no choice but to fall back on channel IDs. We're just lucky that (for now) the IDs are generated predictably. It's true one should not rely on this, but right now one can't do it any other way.