LTTng » LTTng-tools

Quick Update: I tried the same scenario without traffic control and I still see the issue of CTF and metadata files with 0 file size.

Any chance you can run this experiment with logs on the session daemon
side ? (-vvv --versbose-consumer).

I might have found why this issue occurs but confirming it with your
logs would help.

Thanks!

Hi Jesus,

I need more information on a theory I have.

First, can you tell me when the applications are started (before or
after the start) ?

Second, in the example above, I would like you (or someone :) to tell me
what metadata contains for incomplete trace. You should have the file
magic number at the beginning: "57 1d d1 75" and then either some data
or zeroes.

Please, for each incomplete trace can you give me this information
because this would explains a LOT!

Also, the lifetime of the application here is very important as well
meaning that when it ends? and when is it started.

Thanks!

Just in case I was not clear. The 4 apps are never killed. I launch them before I run the trace and I never stopped them.

Ok so with applications running before and after the experiment this is
good. You should have everything since the lttng start will make sure
every registered application is setup before going further.

The problem I see with the logs is that your relayd dies just after the
destroy command and for which after, there is still data needed to be
streamed. I think this is due to the fact that between a stop and a
destroy, the UST tracer can provide more padding... but appart from
that, it does not explain the data file set to 0.

On line 1354, the problem starts. The consumer ask for a close_stream
command which has been initiated by the streams being destroyed on the
consumer side done by the lttng destroy command. This is OK since we are
basically ending the communication but the relayd died before.

So here is my question, in your experiment, when are you killing the
relayd ? I suppose after the "lttng stop" command has been issued and
returned for all nodes?

I experimented with delaying the killing of relayd last night and I still saw the issue of zero file size. I tried with as much as a 30-second delay, which I consider a very long time considering that only one low-throughput event from one app is getting hit for a period of 10 seconds.

For the amount of data being produced by this session, how long do you estimate we need to delay the killing of relayd?