LTTng » LTTng-tools

Does this relayd log fits the above consumer/sessiond log file ?

The sessiond log above shows two different relay daemons... and this log shows NO data pending command from SC-1 node which makes it difficult to correlate both logs if they are from a different test case.

Sorry, didn't realize you still needed both sides of the equation. Full set of logs coming up soon...

Hmmm somethings weird is going on here. The session daemon is asking for data pending on the stream handle 4 that apparently exists but has never received data on the relayd side making the sequence number never initialized making the data pending test failing.

The problem I have is there is NO log at all of the stream handle 4 being created on either sides but it does exists. The stream handle created in this logs begins at 61 ... so I think there maybe a missing session daemon log here ?

Now, I wonder how it is possible for a stream to never have sent data to the relayd thus making the sequence number never initialized. Even though the stream does not produce data, there should be at least some padding sent making at least one packet sent for the stream.

We can also see this problem for "stream id 3" but a packet was clearly sent :

DEBUG3: Receiving data of size 68 for stream id 3 seqnum 0 [in relay_process_data() at mai
946745 16:10:37 01/07/2013 IN TraceC "Previous log message truncated by 2385 character(s)

And this gets repeated twice... so can we get the truncated log because if an error occurred that would make the sequence number not being updated, it's right there. I suspect that there is an error during the recvmsg() or when writing the data on disk which would not update the prev_seq counter.

I might send you a patch in a jiffy to try to fix a possible issue here. In the meantime, I really need the "truncated" part.

Thanks!

Hmmm I'm having problem with the log... there is corrupted data in it at several place. See line 1210 or 1251 ... lot of unknown bytes.... :S

I feel like a lot of information is missing since I don't see some commands that should be there.

relayd coredump data:

(gdb) bt full
#0 0x0000000000405b54 in relay_process_data (streams_ht=<optimized out>, cmd=<optimized out>) at main.c:1796
ret = <optimized out>
stream = 0x616150
data_hdr = {circuit_id = 0, stream_id = 216172782113783808, net_seq_num = 0, data_size = 1140850688, padding_size = 3155099648}
func = "relay_process_data"
PRETTY_FUNCTION = "relay_process_data"
#1 relay_thread_worker (data=<optimized out>) at main.c:2148
revents = <optimized out>
pollfd = <optimized out>
i = 0
seen_control = 0
last_notdel_data_fd = 13
ret = 1
err = <optimized out>
last_seen_data_fd = -1
relay_connection = 0x614fd0
events = {epfd = 10, nb_fd = 10, alloc_size = 16, init_size = 2, events = 0x616d80}
relay_connections_ht = 0x610010
node = 0x614fe8
iter = {iter = {node = 0x614ff0, next = 0x610030}}
streams_ht = 0x6149d0
recv_hdr = {circuit_id = 0, data_size = 581245826907504640, cmd = 16777216, cmd_version = 0}
func = "relay_thread_worker"
#2 0x00007fe262f6e7b6 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#3 0x00007fe262ccac6d in clone () from /lib64/libc.so.6
No symbol table info available.
#4 0x0000000000000000 in ?? ()
No symbol table info available.