Actions
Bug #1358
openFailed to deploy lttng modules on NVIDIA jetson device
Start date:
09/13/2022
Due date:
% Done:
0%
Estimated time:
Description
Hello, I installed lttng and lttng modules on NVIDIA Orin.
When using apt install, the results are as follows.
# lttng list --kernel
Error: Unable to list kernel events: Kernel tracer not available
#ps aux | grep lttng-sessiond
root 52100 0.0 0.0 1022064 12736 ? Ssl 15:05 0:00 /usr/bin/lttng-sessiond
root 52101 0.0 0.0 41968 664 ? S 15:05 0:00 /usr/bin/lttng-sessiond
orin-d 62549 0.0 0.0 11640 684 pts/0 S+ 20:54 0:00 grep --color=auto lttng-sessiond
# dpkg -l | grep lttng ii liblttng-ctl0:arm64 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility library ii liblttng-ust-ctl4:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (trace control library) ii liblttng-ust-dev:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (development files) ii liblttng-ust-python-agent0:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (Python agent native library) ii liblttng-ust0:arm64 2.12.2-1~ubuntu20.04.1 arm64 LTTng 2.0 Userspace Tracer (tracing libraries) ii lttng-modules-dkms 2.12.6-1~ubuntu20.04.1 all Linux Trace Toolkit (LTTng) kernel modules (DKMS) ii lttng-tools 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility programs ii python3-lttng 2.12.4-1~ubuntu20.04.1 arm64 LTTng control and utility Python bindings
The device information is as follows.
# uname -a
Linux orind-d 5.10.65-tegra #2 SMP PREEMPT Thu Jun 16 18:24:26 CST 2022 aarch64 aarch64 aarch64 GNU/Linux
# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
At the same time, I tried the method of source code, such as the source code installation method introduced in https://lttng.org/docs/v2.13/.
# dpkg -l | grep -e libuuid -e popt -e userspace -e libxml2 ii can-utils 2018.02.0-1ubuntu1 arm64 SocketCAN userspace utilities and tools ii dmsetup 2:1.02.167-1ubuntu1 arm64 Linux Kernel Device Mapper userspace library ii gvfs:arm64 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - GIO module ii gvfs-backends 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - backends ii gvfs-bin 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - deprecated command-line tools ii gvfs-common 1.44.1-1ubuntu1 all userspace virtual filesystem - common data files ii gvfs-daemons 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - servers ii gvfs-fuse 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - fuse server ii gvfs-libs:arm64 1.44.1-1ubuntu1 arm64 userspace virtual filesystem - private libraries ii libdevmapper1.02.1:arm64 2:1.02.167-1ubuntu1 arm64 Linux Kernel Device Mapper userspace library ii libi2c0:arm64 4.1-2build2 arm64 userspace I2C programming library ii libibverbs1:arm64 28.0-1ubuntu1 arm64 Library for direct userspace use of RDMA (InfiniBand/iWARP) ii libnftnl11:arm64 1.1.5-1 arm64 Netfilter nftables userspace API library ii libpopt-dev:arm64 1.16-14 arm64 lib for parsing cmdline parameters - development files ii libpopt0:arm64 1.16-14 arm64 lib for parsing cmdline parameters ii liburcu-dev:arm64 0.12.2-1~ubuntu20.04.2 arm64 userspace RCU (read-copy-update) library - development files ii liburcu6:arm64 0.12.2-1~ubuntu20.04.2 arm64 userspace RCU (read-copy-update) library ii libusb-1.0-0:arm64 2:1.0.23-2build1 arm64 userspace USB programming library ii libusb-1.0-0-dev:arm64 2:1.0.23-2build1 arm64 userspace USB programming library development files ii libuuid1:arm64 2.34-0.1ubuntu9.3 arm64 Universally Unique ID library ii libxml2:arm64 2.9.10+dfsg-5ubuntu0.20.04.1 arm64 GNOME XML library ii libxml2-dev:arm64 2.9.10+dfsg-5ubuntu0.20.04.1 arm64 Development files for the GNOME XML library ii libxml2-utils 2.9.10+dfsg-5ubuntu0.20.04.3 arm64 XML utilities ii network-manager 1.22.10-1ubuntu2.3 arm64 network management framework (daemon and userspace tools) ii nvidia-l4t-optee 34.1.0-20220406120854 arm64 OP-TEE userspace daemons, test programs and libraries ii python3-lxml:arm64 4.5.0-1ubuntu0.5 arm64 pythonic binding for the libxml2 and libxslt libraries
sudo ln -snf /usr/src/linux-headers-5.10.65-tegra-ubuntu20.04_aarch64/kernel-5.10 /lib/modules/5.10.65-tegra/build
# orin-d@orind-d:~/tmp/lttng-modules-2.13.5$ make
/home/orin-d/tmp/lttng-modules-2.13.5/src/wrapper/kallsyms.c:20:3: error: #error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
20 | # error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
| ^~~~~
make[2]: *** [scripts/Makefile.build:281: /home/orin-d/tmp/lttng-modules-2.13.5/src/wrapper/kallsyms.o] Error 1
make[1]: *** [Makefile:1852: /home/orin-d/tmp/lttng-modules-2.13.5/src] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.10.65-tegra-ubuntu20.04_aarch64/kernel-5.10'
make: *** [Makefile:31: modules] Error 2
Files
Updated by Mathieu Desnoyers over 1 year ago
You will need to rebuild your kernel with CONFIG_KPROBES=y, as stated in the error message. This is required from kernel 5.7.0 onwards.
Updated by liuhonggang liu over 1 year ago
Hello.
I rebuild my kernel with CONFIG_KPROBES=y.
I rebuild my kernel with CONFIG_KPROBES=y.
- uname -a
Linux orind-d 5.10.65-tegra #2 SMP PREEMPT Thu Jun 16 18:24:26 CST 2022 aarch64 aarch64 aarch64 GNU/Linux - cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
- zcat /proc/config.gz | grep -i kprobes
CONFIG_KPROBES=y
CONFIG_HAVE_KPROBES=y
- error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
How can I solve it please?
The platform I am using is shown below.
Updated by liuhonggang liu over 1 year ago
Hello.
I rebuild my kernel with CONFIG_KPROBES=y.
I rebuild my kernel with CONFIG_KPROBES=y.
- uname -a
Linux orind-d 5.10.65-tegra #2 SMP PREEMPT Thu Jun 16 18:24:26 CST 2022 aarch64 aarch64 aarch64 GNU/Linux - cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
- zcat /proc/config.gz | grep -i kprobes
CONFIG_KPROBES=y
CONFIG_HAVE_KPROBES=y
- error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
How can I solve it please?
The platform I am using is shown below.
Updated by liuhonggang liu over 1 year ago
Hello.
I rebuild my kernel with CONFIG_KPROBES=y.
I rebuild my kernel with CONFIG_KPROBES=y.
- uname -a
Linux orind-d 5.10.65-tegra #2 SMP PREEMPT Thu Jun 16 18:24:26 CST 2022 aarch64 aarch64 aarch64 GNU/Linux - cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
- zcat /proc/config.gz | grep -i kprobes
CONFIG_KPROBES=y
CONFIG_HAVE_KPROBES=y
- error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
How can I solve it please?
The platform I am using is shown below.
Updated by Michael Jeanson over 1 year ago
The output from '/proc/config.gz' confirms that the running kernel does indeed have CONFIG_KPROBE, the lttng-modules-dkms package uses the kernel headers and config located in '/lib/modules/$(uname -r)/build/', those have to match the running kernel.
Updated by liuhonggang liu over 1 year ago
- File 1667362822097.png 1667362822097.png added
I rebuild my kernel with CONFIG_KPROBES=y.
uname -a
Linux inspur 5.10.104 #1 SMP PREEMPT Mon Oct 24 19:42:41 CST 2022 aarch64 aarch64 aarch64 GNU/Linux
sudo apt-get install lttng-modules-dkm
the log is as follows.
Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: babeltrace liblttng-ctl0 liblttng-ust-ctl4 liburcu6 Suggested packages: lttng-modules-dkms The following NEW packages will be installed: babeltrace liblttng-ctl0 liblttng-ust-ctl4 liburcu6 lttng-tools 0 upgraded, 5 newly installed, 0 to remove and 219 not upgraded. Need to get 0 B/1049 kB of archives. After this operation, 4376 kB of additional disk space will be used. Do you want to continue? [Y/n] y debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package liburcu6:arm64. (Reading database ... 148021 files and directories currently installed.) Preparing to unpack .../liburcu6_0.12.2-1~ubuntu20.04.2_arm64.deb ... Unpacking liburcu6:arm64 (0.12.2-1~ubuntu20.04.2) ... Selecting previously unselected package liblttng-ctl0:arm64. Preparing to unpack .../liblttng-ctl0_2.12.4-1~ubuntu20.04.1_arm64.deb ... Unpacking liblttng-ctl0:arm64 (2.12.4-1~ubuntu20.04.1) ... Selecting previously unselected package liblttng-ust-ctl4:arm64. Preparing to unpack .../liblttng-ust-ctl4_2.12.2-1~ubuntu20.04.1_arm64.deb ... Unpacking liblttng-ust-ctl4:arm64 (2.12.2-1~ubuntu20.04.1) ... Selecting previously unselected package lttng-tools. Preparing to unpack .../lttng-tools_2.12.4-1~ubuntu20.04.1_arm64.deb ... Unpacking lttng-tools (2.12.4-1~ubuntu20.04.1) ... Selecting previously unselected package babeltrace. Preparing to unpack .../babeltrace_1.5.8-1build1_arm64.deb ... Unpacking babeltrace (1.5.8-1build1) ... Setting up liblttng-ust-ctl4:arm64 (2.12.2-1~ubuntu20.04.1) ... Setting up babeltrace (1.5.8-1build1) ... Setting up liburcu6:arm64 (0.12.2-1~ubuntu20.04.2) ... Setting up liblttng-ctl0:arm64 (2.12.4-1~ubuntu20.04.1) ... Processing triggers for sgml-base (1.29.1) ... Processing triggers for libc-bin (2.31-0ubuntu9.9) ... Processing triggers for systemd (245.4-4ubuntu3.17) ... Processing triggers for man-db (2.9.1-1) ... Setting up lttng-tools (2.12.4-1~ubuntu20.04.1) ... Created symlink /etc/systemd/system/multi-user.target.wants/lttng-sessiond.service → /lib/systemd/system/lttng-sessiond.service. Processing triggers for systemd (245.4-4ubuntu3.17) ... orin-c@inspur:~$ sudo apt-get install lttng-modules-dkms Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: dkms linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic linux-headers-generic Suggested packages: menu The following NEW packages will be installed: dkms linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic linux-headers-generic lttng-modules-dkms 0 upgraded, 5 newly installed, 0 to remove and 219 not upgraded. Need to get 0 B/12.3 MB of archives. After this operation, 85.1 MB of additional disk space will be used. Do you want to continue? [Y/n] y debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package dkms. (Reading database ... 148110 files and directories currently installed.) Preparing to unpack .../dkms_2.8.1-5ubuntu2_all.deb ... Unpacking dkms (2.8.1-5ubuntu2) ... Selecting previously unselected package linux-headers-5.4.0-131. Preparing to unpack .../linux-headers-5.4.0-131_5.4.0-131.147_all.deb ... Unpacking linux-headers-5.4.0-131 (5.4.0-131.147) ... Selecting previously unselected package linux-headers-5.4.0-131-generic. Preparing to unpack .../linux-headers-5.4.0-131-generic_5.4.0-131.147_arm64.deb ... Unpacking linux-headers-5.4.0-131-generic (5.4.0-131.147) ... Selecting previously unselected package linux-headers-generic. Preparing to unpack .../linux-headers-generic_5.4.0.131.131_arm64.deb ... Unpacking linux-headers-generic (5.4.0.131.131) ... Selecting previously unselected package lttng-modules-dkms. Preparing to unpack .../lttng-modules-dkms_2.12.6-1~ubuntu20.04.1_all.deb ... Unpacking lttng-modules-dkms (2.12.6-1~ubuntu20.04.1) ... Setting up linux-headers-5.4.0-131 (5.4.0-131.147) ... Setting up dkms (2.8.1-5ubuntu2) ... Setting up lttng-modules-dkms (2.12.6-1~ubuntu20.04.1) ... Loading new lttng-modules-2.12.6 DKMS files... It is likely that 5.10.104 belongs to a chroot's host Building for 5.10.104, 5.10.104-tegra and 5.4.0-131-generic Building initial module for 5.10.104 ERROR (dkms apport): kernel package linux-headers-5.10.104 is not supported Error! Bad return status for module build on kernel: 5.10.104 (aarch64) Consult /var/lib/dkms/lttng-modules/2.12.6/build/make.log for more information. dpkg: error processing package lttng-modules-dkms (--configure): installed lttng-modules-dkms package post-installation script subprocess returned error exit status 10 Setting up linux-headers-5.4.0-131-generic (5.4.0-131.147) ... /etc/kernel/header_postinst.d/dkms: * dkms: running auto installation service for kernel 5.4.0-131-generic Kernel preparation unnecessary for this kernel. Skipping... Building module: cleaning build area... make -j4 KERNELRELEASE=5.4.0-131-generic -C /lib/modules/5.4.0-131-generic/build M=/var/lib/dkms/lttng-modules/2.12.6/build modules CONFIG_LTTNG=m CONFIG_LTTNG_CLOCK_PLUGIN_TEST=m.......(bad exit status: 2) ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/lttng-modules-dkms.0.crash' Error! Bad return status for module build on kernel: 5.4.0-131-generic (aarch64) Consult /var/lib/dkms/lttng-modules/2.12.6/build/make.log for more information. ...done. Setting up linux-headers-generic (5.4.0.131.131) ... Processing triggers for man-db (2.9.1-1) ... Errors were encountered while processing: lttng-modules-dkms E: Sub-process /usr/bin/dpkg returned an error code (1)
/var/lib/dkms/lttng-modules/2.12.6/build/make.log. The content is as follows.
DKMS make.log for lttng-modules-2.12.6 for kernel 5.4.0-131-generic (aarch64)
Wed Nov 2 04:04:29 UTC 2022
make: Entering directory '/usr/src/linux-headers-5.4.0-131-generic'
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-client-discard.o
/var/lib/dkms/lttng-modules/2.12.6/build/probes/Kbuild:166: Files ./fs/btrfs/*.h not found. Probe "btrfs" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.12.6/build/probes/Kbuild:177: Files ./fs/ext4/*.h not found. Probe "ext4" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.12.6/build/probes/Kbuild:208: File ./drivers/base/regmap/trace.h not found. Probe "regmap" is disabled. Need Linux 4.1+ kernel source tree to enable it.
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_backend.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/tests/probes/lttng-test.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-sched.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/tests/clock-plugin/lttng-clock-plugin-test.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_frontend.o
LD [M] /var/lib/dkms/lttng-modules/2.12.6/build/tests/lttng-test.o
LD [M] /var/lib/dkms/lttng-modules/2.12.6/build/tests/lttng-clock-plugin-test.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_iterator.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_vfs.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-irq.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-timer.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-kmem.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-client-overwrite.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_splice.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/ringbuffer/ring_buffer_mmap.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/prio_heap/lttng_prio_heap.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-module.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-power.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-metadata-client.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/../wrapper/splice.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-statedump.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-9p.o
LD [M] /var/lib/dkms/lttng-modules/2.12.6/build/lib/lttng-lib-ring-buffer.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-client-mmap-discard.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-i2c.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-kvm.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-signal.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-block.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-client-mmap-overwrite.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-ring-buffer-metadata-mmap-client.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-clock.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-napi.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-skb.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-net.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-events.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-abi.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-string-utils.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-sock.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-probes.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-udp.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-pid.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-asoc.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-procname.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-gpio.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-jbd2.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-regulator.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-scsi.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-vmscan.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-prio.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-nice.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-vpid.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-compaction.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-tid.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-vtid.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-printk.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-ppid.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-random.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-rcu.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-vppid.o
/var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-random.c:18:10: fatal error: trace/events/random.h: No such file or directory
18 | #include <trace/events/random.h>
| ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [scripts/Makefile.build:270: /var/lib/dkms/lttng-modules/2.12.6/build/probes/lttng-probe-random.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-cpu-id.o
CC [M] /var/lib/dkms/lttng-modules/2.12.6/build/lttng-context-uid.o
make[1]: *** [scripts/Makefile.build:519: /var/lib/dkms/lttng-modules/2.12.6/build/probes] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:1762: /var/lib/dkms/lttng-modules/2.12.6/build] Error 2
make: Leaving directory '/usr/src/linux-headers-5.4.0-131-generic'
According to the above log, it looks like the cause of the problem may be that the kernel headers for this kernel does not seem to be installed. And the header files in the following directory are being used /usr/src/linux-headers-5.4.0-131-generic.
From the image below, there is no linux-headers-5.10.104 in /usr/src.
Is this the cause of the problem? Or do you know the root cause of the above problem
Updated by Michael Jeanson over 1 year ago
There is 2 different issues here, the lttng-modules-dkms will by default only build modules for packaged kernels. In this case the build failure you see in the dkms.log is for the '5.4.0-131-generic' kernel shipped by ubuntu.
The second issue is that you don't seem to have the headers for your '5.10.104' kernel, there is a dangling symlink for the 'source' directory but the headers we are interrested in should be in '/lib/modules/5.10.104/build'. This directory exists but I don't see what its content is.
If you can confirm that the kernel headers are indeed installed, the next step would be to try building lttng-modules from the latest 2.12.x source tarball.
Updated by liuhonggang liu over 1 year ago
Install lttng-modules with the correct version of the header files.
orin-c@inspur:/lib/modules/5.10.104-tegra$ ll total 2012 drwxr-xr-x 4 root root 4096 Oct 24 11:32 ./ drwxr-xr-x 3 root root 4096 Nov 3 02:35 ../ lrwxrwxrwx 1 root root 69 Aug 11 03:37 build -> /usr/src/linux-headers-5.10.104-tegra-ubuntu20.04_aarch64/kernel-5.10/ drwxr-xr-x 3 root root 4096 Oct 24 11:31 extra/ drwxr-xr-x 9 root root 4096 Oct 24 11:31 kernel/ -rw-r--r-- 1 root root 506756 Oct 24 11:32 modules.alias -rw-r--r-- 1 root root 497516 Oct 24 11:32 modules.alias.bin -rw-r--r-- 1 root root 27603 Aug 11 03:37 modules.builtin -rw-r--r-- 1 root root 50099 Oct 24 11:32 modules.builtin.alias.bin -rw-r--r-- 1 root root 30111 Oct 24 11:32 modules.builtin.bin -rw-r--r-- 1 root root 154311 Aug 11 03:37 modules.builtin.modinfo -rw-r--r-- 1 root root 88190 Oct 24 11:32 modules.dep -rw-r--r-- 1 root root 138538 Oct 24 11:32 modules.dep.bin -rw-r--r-- 1 root root 187 Oct 24 11:32 modules.devname -rw-r--r-- 1 root root 45829 Aug 11 03:37 modules.order -rw-r--r-- 1 root root 786 Oct 24 11:32 modules.softdep -rw-r--r-- 1 root root 210396 Oct 24 11:32 modules.symbols -rw-r--r-- 1 root root 261524 Oct 24 11:32 modules.symbols.bin
Install lttng-modules from source code.
orin-c@inspur:~/lhg/lttng-modules-2.12.10$ make
make -C /lib/modules/5.10.104-tegra/build M=/home/orin-c/lhg/lttng-modules-2.12.10 \
CONFIG_LTTNG=m CONFIG_LTTNG_CLOCK_PLUGIN_TEST=m \
KCPPFLAGS='' \
modules
make[1]: Entering directory '/usr/src/linux-headers-5.10.104-tegra-ubuntu20.04_aarch64/kernel
/home/orin-c/lhg/lttng-modules-2.12.10/probes/Kbuild:166: Files ./fs/btrfs/*.h not found. Pro
/home/orin-c/lhg/lttng-modules-2.12.10/probes/Kbuild:179: Files ./fs/ext4/*.h not found. Prob
/home/orin-c/lhg/lttng-modules-2.12.10/probes/Kbuild:208: File ./drivers/base/regmap/trace.h enable it.
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-sched.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-irq.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-timer.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-kmem.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-module.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-power.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-statedump.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-9p.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-i2c.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-kvm.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-signal.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-block.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-napi.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-skb.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-net.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-sock.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-udp.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-asoc.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-gpio.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-jbd2.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-regulator.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-scsi.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-vmscan.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-compaction.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-printk.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-random.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-rcu.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-sunrpc.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-v4l2.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-workqueue.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-writeback.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_backend.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_frontend.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_iterator.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_vfs.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_splice.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/ringbuffer/ring_buffer_mmap.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/prio_heap/lttng_prio_heap.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/../wrapper/splice.o
LD [M] /home/orin-c/lhg/lttng-modules-2.12.10/lib/lttng-lib-ring-buffer.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/tests/probes/lttng-test.o
LD [M] /home/orin-c/lhg/lttng-modules-2.12.10/tests/lttng-test.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/tests/clock-plugin/lttng-clock-plugin-test.o
LD [M] /home/orin-c/lhg/lttng-modules-2.12.10/tests/lttng-clock-plugin-test.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-client-discard.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-client-overwrite.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-metadata-client.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-client-mmap-discard.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-client-mmap-overwrite.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-ring-buffer-metadata-mmap-client.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-clock.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-events.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-abi.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-string-utils.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-probes.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-pid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-procname.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-prio.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-nice.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vpid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-tid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vtid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-ppid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vppid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-cpu-id.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-uid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-euid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-suid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-gid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-egid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-sgid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vuid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-veuid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vsuid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vgid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vegid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-vsgid.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-interruptible.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-need-reschedule.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-callstack.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-calibrate.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-hostname.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-tracker-id.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-filter.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-filter-interpreter.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-filter-specialize.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-filter-validator.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/probes/lttng-probe-user.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-tp-mempool.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-syscalls.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-perf-counters.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-preemptible.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-tracepoint.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-cgroup-ns.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-ipc-ns.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-net-ns.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-pid-ns.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-user-ns.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-context-uts-ns.o
LD [M] /home/orin-c/lhg/lttng-modules-2.12.10/lttng-tracer.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/wrapper/page_alloc.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/wrapper/random.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/wrapper/trace-clock.o
CC [M] /home/orin-c/lhg/lttng-modules-2.12.10/wrapper/kallsyms.o
/home/orin-c/lhg/lttng-modules-2.12.10/wrapper/kallsyms.c:20:3: error: #error "LTTng-modules
20 | # error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
| ^~~~~
make[2]: *** [scripts/Makefile.build:281: /home/orin-c/lhg/lttng-modules-2.12.10/wrapper/kall
make[1]: *** [Makefile:1854: /home/orin-c/lhg/lttng-modules-2.12.10] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.10.104-tegra-ubuntu20.04_aarch64/kernel-
make: *** [Makefile:162: modules] Error 2
zcat /proc/config.gz | grep -i kprobes CONFIG_KPROBES=y CONFIG_HAVE_KPROBES=y
The error still appears in the log, how can I go deeper to find the cause of the error?
Updated by Michael Jeanson over 1 year ago
The kernel headers in '/lib/modules/5.10.104-tegra/build' need to be configured with the same '.config' file as the running kernel.
Updated by liuhonggang liu over 1 year ago
Hello.
I’m running the last kernel available from JetPack5.0.1 Linux for Tegra r35.1, which is a LTS ubuntu 20.04 LTS. Install lttng-modules-2.12.10 from source on this device. I confirm that the kernel headers are indeed installed and CONFIG_KPROBES = y on my system.
1. Does lttng-modules support installation on this system?
2. If installation is supported, can you reproduce the following error
# error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"
3. If you do not have the conditions to reproduce, I can support you in every step of analysis and give you feedback on the results of every step.
Updated by liuhonggang liu over 1 year ago
Hello.
I’m running the last kernel available from JetPack5.0.1 Linux for Tegra r35.1, which is a LTS ubuntu 20.04 LTS. Install lttng-modules-2.12.10 from source on this device. I confirm that the kernel headers are indeed installed and CONFIG_KPROBES = y on my system.
1. Does lttng-modules support installation on this system?
2. If installation is supported, can you reproduce the following error
error "LTTng-modules requires CONFIG_KPROBES on kernels >= 5.7.0"3. If you do not have the conditions to reproduce, I can support you in every step of analysis and give you feedback on the results of every step.
Updated by Michael Jeanson over 1 year ago
I downloaded the 5.10.104 kernel sources from Nvidia on this page:
https://developer.nvidia.com/embedded/jetson-linux-r351Then used the defconfig to which I added CONFIG_KPROBES and had no issues building lttng-modules '2.12' and '2.13'.
I'm still convinced your issue is that the headers / kernel sources which you use to build lttng-modules don't match you running kernel.
We do offer professional services at "https://efficios.com", including packaging and deployment, it might be an option for your organisation.
Updated by liuhonggang liu over 1 year ago
We use JETSON AGX ORIN, the .config file shows no problem when flashing, and the kernel compilation and flashing are no problem.
1. Did you modify the kernel before compiling? If so, how did you compile lttng-modules-dkms into the kernel?
2. Have you tested installing lttng-modules-dkms on orin devkit? Is it convenient to show your environment here?
3. How much does it cost via "https://efficios.com"?
4. Would you like to introduce how you used https://developer.nvidia.com/embedded/jetson-linux-r351 to install lttng-module-dkms?
Your answer will give me a great help, looking forward to your answer.
Thank you.
Updated by liuhonggang liu over 1 year ago
We use JETSON AGX ORIN, the .config file shows no problem when flashing, and the kernel compilation and flashing are no problem.
1. Did you modify the kernel before compiling? If so, how did you compile lttng-modules-dkms into the kernel?
2. Have you tested installing lttng-modules-dkms on orin devkit? Is it convenient to show your environment here?
3. How much does it cost via "https://efficios.com"?
4. Would you like to introduce how you used https://developer.nvidia.com/embedded/jetson-linux-r351 to install lttng-module-dkms?
Your answer will give me a great help, looking forward to your answer.
Thank you.
Updated by Michael Jeanson over 1 year ago
liuhonggang liu wrote in #note-14:
We use JETSON AGX ORIN, the .config file shows no problem when flashing, and the kernel compilation and flashing are no problem.
1. Did you modify the kernel before compiling? If so, how did you compile lttng-modules-dkms into the kernel?
No, I built a kernel from the Nvidia source tarball and then built lttng-modules also from a source tarball.
2. Have you tested installing lttng-modules-dkms on orin devkit? Is it convenient to show your environment here?
No
3. How much does it cost via "https://efficios.com"?
It really depends on the scope of the support you want, you can contact "info@efficios.com".
4. Would you like to introduce how you used https://developer.nvidia.com/embedded/jetson-linux-r351 to install lttng-module-dkms?
Your answer will give me a great help, looking forward to your answer.
Thank you.
Updated by liuhonggang liu over 1 year ago
thank you.
Can lttng-modules-dkms be deployed and used on nvidia's tegra platform?
Actions