Project

General

Profile

Actions

Bug #1345

open

NULL pointer dereference when adding perf:cpu:cache-misses context

Added by Francis Deslauriers 6 months ago. Updated 6 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
01/10/2022
Due date:
% Done:

0%

Estimated time:

Description

I witnessed a NULL pointer dereference when testing a feature I am working on.

The NULL deref occurs when adding a perf event context.
This occurs on my Qemu VM with a single CPU with built-in LTTng modules.I haven't tested with regular loadable modules at this time. I attached the .config to this issue.

In the following code the pevent variable is NULL when calling perf_event_release_kernel()

       142  int lttng_cpuhp_perf_counter_dead(unsigned int cpu,   
│      143                  struct lttng_cpuhp_node *node)     
│      144  {   
│      145          struct lttng_perf_counter_field *perf_field =                   
│      146                  container_of(node, struct lttng_perf_counter_field,        
│      147                                  cpuhp_prepare);                        
│      148          struct perf_event **events = perf_field->e;               
│      149          struct perf_event *pevent;                              
│      150                                                                        
│      151          pevent = events[cpu];                                               
│      152          events[cpu] = NULL;                                          
│      153          barrier();      /* NULLify event before perf counter teardown */ 
│  >   154          perf_event_release_kernel(pevent);                    
│      155          return 0;                                            
│      156  }                                          
│      157             

Commit used:

commit 45fe4e1a42028b821757e0b98f9b33bf435a108a (HEAD -> master, origin/HEAD)
Author: Michael Jeanson <mjeanson@efficios.com>
Date:   Tue Dec 14 14:44:35 2021 -0500

    fix: mm: move kvmalloc-related functions to slab.h (v5.16)

GDB backtrace:

#0  dump_stack () at lib/dump_stack.c:89
#1  0xffffffff822ea240 in __kasan_report (ip=<optimized out>, is_write=false, size=8, addr=544) at mm/kasan/report.c:549
#2  kasan_report (addr=544, size=size@entry=8, is_write=is_write@entry=false, ip=<optimized out>) at mm/kasan/report.c:562
#3  0xffffffff818b1429 in check_memory_region_inline (ret_ip=<optimized out>, write=false, size=8, addr=544) at mm/kasan/generic.c:186
#4  __asan_load8 (addr=addr@entry=544) at mm/kasan/generic.c:252
#5  0xffffffff8148fd3c in perf_event_release_kernel (event=0x0 <fixed_percpu_data>) at kernel/events/core.c:4989
#6  0xffffffff8171e5a9 in lttng_cpuhp_perf_counter_dead (cpu=0, node=0xffff88800d55c180) at lttng/src/lttng-context-perf-counters.c:154
#7  0xffffffff814bb4e5 in lttng_hotplug_dead (cpu=cpu@entry=0, node=node@entry=0xffff88800d55c188) at lttng/src/lttng-events.c:4076
#8  0xffffffff81206dc9 in cpuhp_invoke_callback (cpu=cpu@entry=0, state=state@entry=67, bringup=<optimized out>, node=node@entry=0xffff88800d55c188, lastp=lastp@entry=0x0 <fixed_percpu_data>) at kernel/cpu.c:185
#9  0xffffffff812079ef in cpuhp_issue_call (cpu=0, state=<optimized out>, bringup=<optimized out>, node=0xffff88800d55c188) at kernel/cpu.c:1777
#10 0xffffffff81207c29 in __cpuhp_state_remove_instance (state=67, node=0xffff88800d55c188, invoke=<optimized out>) at kernel/cpu.c:1984
#11 0xffffffff8171dec5 in cpuhp_state_remove_instance (state=67, node=0xffff88800d55c188) at ./include/linux/cpuhotplug.h:389
#12 0xffffffff8171ea44 in lttng_add_perf_counter_to_ctx (type=0, config=3, name=0xffff88800b8afa90 "perf_cpu_cache_misses", ctx=0xffff8881071b0588) at lttng/src/lttng-context-perf-counters.c:340
#13 0xffffffff814c5f3f in lttng_abi_add_context (context_param=context_param@entry=0xffff88800b8afa70, ctx=ctx@entry=0xffff8881071b0588, session=<optimized out>, file=0xffff888105f9e780) at lttng/src/lttng-abi.c:288
#14 0xffffffff814c6aaf in lttng_channel_ioctl (file=0xffff888105f9e780, cmd=<optimized out>, arg=<optimized out>) at lttng/src/lttng-abi.c:2525
#15 0xffffffff8192870f in vfs_ioctl (arg=140611793466688, cmd=<optimized out>, filp=0xffff888105f9e780) at fs/ioctl.c:48
#16 __do_sys_ioctl (arg=140611793466688, cmd=1093990001, fd=<optimized out>) at fs/ioctl.c:753
#17 __se_sys_ioctl (arg=140611793466688, cmd=1093990001, fd=<optimized out>) at fs/ioctl.c:739
#18 __x64_sys_ioctl (regs=<optimized out>) at fs/ioctl.c:739
#19 0xffffffff8230bd38 in do_syscall_64 (nr=<optimized out>, regs=0xffff88800b8aff58) at arch/x86/entry/common.c:46
#20 0xffffffff82400068 in entry_SYSCALL_64 () at arch/x86/entry/entry_64.S:120

Commands used:

#!/bin/bash -x

lttng create --output=/tmp/allo

lttng add-context -k --type=perf:cpu:cache-misses

Syslog report:

[   66.995828] BUG: KASAN: null-ptr-deref in perf_event_release_kernel+0x7c/0x7b0
[   66.997302] Read of size 8 at addr 0000000000000220 by task Client manageme/409
[   66.998757]
[   66.999971] ==================================================================
[   67.007641] BUG: kernel NULL pointer dereference, address: 0000000000000220
[   67.008783] #PF: supervisor read access in kernel mode
[   67.009444] #PF: error_code(0x0000) - not-present page
[   67.010073] PGD 0 P4D 0
[   67.010433] Oops: 0000 [#1] SMP KASAN
[   67.011498] CPU: 0 PID: 409 Comm: Client manageme Tainted: G    B             5.10.36+ #737
[   67.013724] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[   67.015989] RIP: 0010:perf_event_release_kernel+0x80/0x7b0
[   67.016882] Code: 88 c0 fc 48 81 c7 00 f1 f1 f1 f1 c7 40 04 00 00 f3 f3 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 e8 84 16 42 00 48 8d 40
[   67.020782] RSP: 0018:ffff88810b24f698 EFLAGS: 00010286
[   67.021829] RAX: ffff88810b24f6e0 RBX: ffff8881092bf848 RCX: dffffc0000000000
[   67.023262] RDX: 0000000000000007 RSI: 0000000000000004 RDI: ffffffff8232616f
[   67.024467] RBP: ffff88810b24f748 R08: 0000000000000000 R09: 0000000000000000
[   67.025816] R10: ffffffff837066c3 R11: fffffbfff06e0cd8 R12: 0000000000000000
[   67.027265] R13: 0000000000000000 R14: ffffffff832fc958 R15: ffffffff832fc958
[   67.028857] FS:  00007f1d7d7fa700(0000) GS:ffff88810c800000(0000) knlGS:0000000000000000
[   67.030572] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   67.032386] CR2: 00007f8e0fac1030 CR3: 000000010aac4000 CR4: 00000000000006f0
[   67.033911] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   67.035143] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   67.035980] Call Trace:
[   67.036283]  ? __might_sleep+0x72/0xd0
[   67.036748]  ? __perf_event_exit_context+0xa0/0xa0
[   67.037392]  ? __kasan_check_write+0x14/0x20
[   67.038095]  lttng_cpuhp_perf_counter_dead+0x97/0xa3
[   67.039034]  lttng_hotplug_dead+0x55/0x60
[   67.039749]  cpuhp_invoke_callback+0x219/0xaf0
[   67.040543]  ? lttng_hotplug_online+0x80/0x80
[   67.041288]  cpuhp_issue_call+0x26f/0x2a0
[   67.042000]  __cpuhp_state_remove_instance+0x159/0x2d0
[   67.042931]  cpuhp_state_remove_instance+0x25/0x27
[   67.043774]  lttng_add_perf_counter_to_ctx+0x48f/0x558
[   67.044903]  ? lttng_cpuhp_perf_counter_dead+0xa3/0xa3
[   67.045769]  ? wrapper_perf_event_create_kernel_counter+0x36/0x36
[   67.046589]  ? perf_counter_get_size+0x43/0x43
[   67.048025]  ? overflow_callback+0x1c/0x1c
[   67.049264]  lttng_abi_add_context.isra.0+0x2df/0x440
[   67.050733]  lttng_channel_ioctl+0x27f/0x660
[   67.051656]  ? __kasan_check_read+0x11/0x20
[   67.052393]  ? lttng_abi_create_stream_fd.isra.0+0x80/0x80
[   67.053071]  ? check_chain_key+0x1e7/0x2d0
[   67.053561]  ? __lock_acquire+0x974/0x3060
[   67.054048]  ? __kasan_check_read+0x11/0x20
[   67.054546]  ? check_chain_key+0x1e7/0x2d0
[   67.055034]  ? register_lock_class+0xcc0/0xcc0
[   67.055577]  ? rcu_read_lock_sched_held+0xa1/0xd0
[   67.056172]  ? check_chain_key+0x1e7/0x2d0
[   67.056685]  ? find_held_lock+0x8e/0xa0
[   67.057171]  ? do_vfs_ioctl+0x529/0x9e0
[   67.057684]  ? __fget_files+0x13e/0x220
[   67.058199]  ? ioctl_file_clone+0xe0/0xe0
[   67.058728]  ? lock_downgrade+0x3c0/0x3c0
[   67.059312]  ? rcu_read_lock_held+0xa1/0xb0
[   67.059841]  ? rcu_read_lock_sched_held+0xd0/0xd0
[   67.060443]  ? __fget_files+0x15d/0x220
[   67.061355]  ? __fget_light+0xec/0x100
[   67.062102]  __x64_sys_ioctl+0xaf/0xf0
[   67.063007]  do_syscall_64+0x38/0x90
[   67.063991]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   67.065443] RIP: 0033:0x7f1d888545d7
[   67.066423] Code: b3 66 90 48 8b 05 b1 48 2d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 08
[   67.070462] RSP: 002b:00007f1d7d7e3d88 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[   67.071522] RAX: ffffffffffffffda RBX: 00007f1d74002fc0 RCX: 00007f1d888545d7
[   67.072711] RDX: 00007f1d74002fc0 RSI: 000000004134f671 RDI: 0000000000000045
[   67.073858] RBP: 000055ac13574700 R08: 0000000000000045 R09: 0000000000000000
[   67.074750] R10: 00007f1d740008d0 R11: 0000000000000246 R12: 000055ac135746c8
[   67.075606] R13: 000055ac135746c8 R14: 00007f1d74002b30 R15: 00007f1d74010210
[   67.076460] Modules linked in:
[   67.076926] CR2: 0000000000000220
[   67.077648] ---[ end trace 795530de3d4e3275 ]---
[   67.078769] RIP: 0010:perf_event_release_kernel+0x80/0x7b0
[   67.079618] Code: 88 c0 fc 48 81 c7 00 f1 f1 f1 f1 c7 40 04 00 00 f3 f3 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 e8 84 16 42 00 48 8d 40
[   67.085113] RSP: 0018:ffff88810b24f698 EFLAGS: 00010286
[   67.086048] RAX: ffff88810b24f6e0 RBX: ffff8881092bf848 RCX: dffffc0000000000
[   67.087196] RDX: 0000000000000007 RSI: 0000000000000004 RDI: ffffffff8232616f
[   67.088396] RBP: ffff88810b24f748 R08: 0000000000000000 R09: 0000000000000000
[   67.089555] R10: ffffffff837066c3 R11: fffffbfff06e0cd8 R12: 0000000000000000
[   67.090959] R13: 0000000000000000 R14: ffffffff832fc958 R15: ffffffff832fc958
[   67.092386] FS:  00007f1d7d7fa700(0000) GS:ffff88810c800000(0000) knlGS:0000000000000000
[   67.093862] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   67.094962] CR2: 00007f8e0fac1030 CR3: 000000010aac4000 CR4: 00000000000006f0
[   67.096552] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   67.098505] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400


Files

.config (71.7 KB) .config Linux config file Francis Deslauriers, 01/10/2022 04:10 PM
Actions #1

Updated by Francis Deslauriers 6 months ago

  • Description updated (diff)
Actions

Also available in: Atom PDF