duff/android_kernel_motorola_sm6225
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
886,168 Commits 2 Branches 0 Tags
c212b2f5d237b894d3a0affb88d0899f3bd8b92f
Commit Graph
886168 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Elver c212b2f5d2 FROMGIT: kfence: use power-efficient work queue to run delayed work 
Use the power-efficient work queue, to avoid the pathological case where
we keep pinning ourselves on the same possibly idle CPU on systems that
want to be power-efficient (https://lwn.net/Articles/731052/).

Link: https://lkml.kernel.org/r/20210421105132.3965998-4-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Hillf Danton <hdanton@sina.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Bug: 172317151
Bug: 185280916
Test: power team confirmed there's no regression
(cherry picked from commit e5bcc059aa2321383325e98628d65258bdcb306f
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I53a2869987cdac165840e3878ff73735a03438fa
 2026-04-28 18:54:34 +00:00
Marco Elver 83058cf90a FROMGIT: kfence: maximize allocation wait timeout duration 
The allocation wait timeout was initially added because of warnings due to
CONFIG_DETECT_HUNG_TASK=y [1].  While the 1 sec timeout is sufficient to
resolve the warnings (given the hung task timeout must be 1 sec or larger)
it may cause unnecessary wake-ups if the system is idle.  [1]
https://lkml.kernel.org/r/CADYN=9J0DQhizAGB0-jz4HOBBh+05kMBXb4c0cXMS7Qi5NAJiw@mail.gmail.com

Fix it by computing the timeout duration in terms of the current
sysctl_hung_task_timeout_secs value.

Link: https://lkml.kernel.org/r/20210421105132.3965998-3-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Hillf Danton <hdanton@sina.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Bug: 172317151
Bug: 185280916
Test: power team confirmed there's no regression
(cherry picked from commit 5d744ff5acbb40f3b8f1703cb433432889f88fbd
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I1a8da1844aaded4827bda904ca621a27bbead4bc
 2026-04-28 18:54:34 +00:00
Marco Elver f884d4a99b FROMGIT: kfence: await for allocation using wait_event 
Patch series "kfence: optimize timer scheduling", v2.

We have observed that mostly-idle systems with KFENCE enabled wake up
otherwise idle CPUs, preventing such to enter a lower power state.
Debugging revealed that KFENCE spends too much active time in
toggle_allocation_gate().

While the first version of KFENCE was using all the right bits to be
scheduling optimal, and thus power efficient, by simply using wait_event()
+ wake_up(), that code was unfortunately removed.

As KFENCE was exposed to various different configs and tests, the
scheduling optimal code slowly disappeared.  First because of hung task
warnings, and finally because of deadlocks when an allocation is made by
timer code with debug objects enabled.  Clearly, the "fixes" were not too
friendly for devices that want to be power efficient.

Therefore, let's try a little harder to fix the hung task and deadlock
problems that we have with wait_event() + wake_up(), while remaining as
scheduling friendly and power efficient as possible.

Crucially, we need to defer the wake_up() to an irq_work, avoiding any
potential for deadlock.

The result with this series is that on the devices where we observed a
power regression, power usage returns back to baseline levels.

This patch (of 3):

On mostly-idle systems, we have observed that toggle_allocation_gate() is
a cause of frequent wake-ups, preventing an otherwise idle CPU to go into
a lower power state.

A late change in KFENCE's development, due to a potential deadlock [1],
required changing the scheduling-friendly wait_event_timeout() and
wake_up() to an open-coded wait-loop using schedule_timeout().  [1]
https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com

To avoid unnecessary wake-ups, switch to using wait_event_timeout().

Unfortunately, we still cannot use a version with direct wake_up() in
__kfence_alloc() due to the same potential for deadlock as in [1].
Instead, add a level of indirection via an irq_work that is scheduled if
we determine that the kfence_timer requires a wake_up().

Link: https://lkml.kernel.org/r/20210421105132.3965998-1-elver@google.com
Link: https://lkml.kernel.org/r/20210421105132.3965998-2-elver@google.com
Fixes: 0ce20dd84089 ("mm: add Kernel Electric-Fence infrastructure")
Signed-off-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Hillf Danton <hdanton@sina.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Bug: 172317151
Bug: 185280916
Test: power team confirmed there's no regression
(cherry picked from commit 0ac66fbbadde5547db190e9d87ff2e77d245f9a2
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Idba39683f0b76eb3f70df1113e43d94845fab5bf
 2026-04-28 18:54:34 +00:00
Marco Elver d4eabce327 FROMGIT: kfence: zero guard page after out-of-bounds access 
After an out-of-bounds accesses, zero the guard page before re-protecting
in kfence_guarded_free().  On one hand this helps make the failure mode of
subsequent out-of-bounds accesses more deterministic, but could also
prevent certain information leaks.

Link: https://lkml.kernel.org/r/20210312121653.348518-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Bug: 172317151
Test: build and run on an ARM64 device
(cherry picked from commit 35eaef74e16dcc85b0659db1c7b4a8d83f7a34ef
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I2bad112b6cbf457892b1dd1a90b909cd88eee2c6
 2026-04-28 18:54:34 +00:00
Marco Elver 484a723c10 UPSTREAM: kfence: make compatible with kmemleak 
Because memblock allocations are registered with kmemleak, the KFENCE
pool was seen by kmemleak as one large object.  Later allocations
through kfence_alloc() that were registered with kmemleak via
slab_post_alloc_hook() would then overlap and trigger a warning.
Therefore, once the pool is initialized, we can remove (free) it from
kmemleak again, since it should be treated as allocator-internal and be
seen as "free memory".

The second problem is that kmemleak is passed the rounded size, and not
the originally requested size, which is also the size of KFENCE objects.
To avoid kmemleak scanning past the end of an object and trigger a
KFENCE out-of-bounds error, fix the size if it is a KFENCE object.

For simplicity, to avoid a call to kfence_ksize() in
slab_post_alloc_hook() (and avoid new IS_ENABLED(CONFIG_DEBUG_KMEMLEAK)
guard), just call kfence_ksize() in mm/kmemleak.c:create_object().

Link: https://lkml.kernel.org/r/20210317084740.3099921-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reported-by: Luis Henriques <lhenriques@suse.de>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Luis Henriques <lhenriques@suse.de>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Bug: 172317151
Test: build and run on an ARM64 device
(cherry picked from commit 9551158069ba8fcc893798d42dc4f978b62ef60f)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ida4d747d3b81b5e8a6f1047b864da89e8e110e61
 2026-04-28 18:54:34 +00:00
Marco Elver 6dadba5f6c UPSTREAM: kfence: fix reports if constant function prefixes exist 
Some architectures prefix all functions with a constant string ('.' on
ppc64).  Add ARCH_FUNC_PREFIX, which may optionally be defined in
<asm/kfence.h>, so that get_stack_skipnr() can work properly.

Link: https://lkml.kernel.org/r/f036c53d-7e81-763c-47f4-6024c6c5f058@csgroup.eu
Link: https://lkml.kernel.org/r/20210304144000.1148590-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reported-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Tested-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Bug: 172317151
Test: build and run on an ARM64 device
(cherry picked from commit 0aa41cae92c1e2e61ae5b3a2dde8e674172e40ac)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iad99dd75be77c26359723d1336e2d1b25beb5a26
 2026-04-28 18:54:34 +00:00
Marco Elver 6198ef7cdd UPSTREAM: kfence: fix printk format for ptrdiff_t 
Use %td for ptrdiff_t.

Link: https://lkml.kernel.org/r/3abbe4c9-16ad-c168-a90f-087978ccd8f7@csgroup.eu
Link: https://lkml.kernel.org/r/20210303121157.3430807-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reported-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Dmitriy Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Bug: 172317151
Test: build and run on an ARM64 device
(cherry picked from commit 702b16d724a61cb97461f403d7a2da29324471b3)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I49c7a8ed6dfecceafd4e25e979c6dd58b1d2dbe2
 2026-04-28 18:54:34 +00:00
Marco Elver 0bcabb04b2 UPSTREAM: kfence: report sensitive information based on no_hash_pointers 
We cannot rely on CONFIG_DEBUG_KERNEL to decide if we're running a "debug
kernel" where we can safely show potentially sensitive information in the
kernel log.

Instead, simply rely on the newly introduced "no_hash_pointers" to print
unhashed kernel pointers, as well as decide if our reports can include
other potentially sensitive information such as registers and corrupted
bytes.

Link: https://lkml.kernel.org/r/20210223082043.1972742-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Cc: Timur Tabi <timur@kernel.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Bug: 177201466
Bug: 180086542
(cherry picked from commit 35beccf0926d42ee0d56e41979ec8cdf814c4769)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ie76403ee1b902742a6ac300ad13bd117a02f93f1
 2026-04-28 18:54:34 +00:00
Timur Tabi fe718b03ac UPSTREAM: lib/vsprintf: no_hash_pointers prints all addresses as unhashed 
If the no_hash_pointers command line parameter is set, then
printk("%p") will print pointers as unhashed, which is useful for
debugging purposes.  This change applies to any function that uses
vsprintf, such as print_hex_dump() and seq_buf_printf().

A large warning message is displayed if this option is enabled.
Unhashed pointers expose kernel addresses, which can be a security
risk.

Also update test_printf to skip the hashed pointer tests if the
command-line option is set.

Signed-off-by: Timur Tabi <timur@kernel.org>
Acked-by: Petr Mladek <pmladek@suse.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Marco Elver <elver@google.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20210214161348.369023-4-timur@kernel.org

Bug: 177201466
Bug: 180086542
(cherry picked from commit 5ead723a20e0447bc7db33dc3070b420e5f80aa6)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I9680bb5e076da9f766a66e6076e5c39291188e40
 2026-04-28 18:54:34 +00:00
Timur Tabi b466062109 UPSTREAM: lib: use KSTM_MODULE_GLOBALS macro in kselftest drivers 
Instead of defining the total/failed test counters manually,
test drivers that are clients of kselftest should use the
macro created for this purpose.

Signed-off-by: Timur Tabi <timur@kernel.org>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Acked-by: Marco Elver <elver@google.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20210214161348.369023-2-timur@kernel.org

Bug: 177201466
Bug: 180086542
(cherry picked from commit 4e89a78779647ca7ee2967551c599633fe9d3647)
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I5d60024e8214548ef1ee3a4e48b0dbc50a8adde9
 2026-04-28 18:54:34 +00:00
Alexander Potapenko 2596d68c80 BACKPORT: kasan: use error_report_end tracepoint 
Make it possible to trace KASAN error reporting.  A good usecase is
watching for trace events from the userspace to detect and process memory
corruption reports from the kernel.

Link: https://lkml.kernel.org/r/20210121131915.1331302-4-glider@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Suggested-by: Marco Elver <elver@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 99bbc9fba9b0c8956b817dd21ecb510da8b676dc
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: resolved conflicts in mm/kasan/report.c]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I60ca062373d13f08ed4ed05bee341a8c28619407
 2026-04-28 18:54:34 +00:00
Alexander Potapenko 415bb6b4c4 FROMGIT: kfence: use error_report_end tracepoint 
Make it possible to trace KFENCE error reporting.  A good usecase is
watching for trace events from the userspace to detect and process memory
corruption reports from the kernel.

Link: https://lkml.kernel.org/r/20210121131915.1331302-3-glider@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Suggested-by: Marco Elver <elver@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 2eb9559621418b844b29d2b56d9962ac019f600b
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I991cd1eee48d107bcdf5e1c0d9194e6a386b9bdf
 2026-04-28 18:54:34 +00:00
Alexander Potapenko d3b772dc9a FROMGIT: tracing: add error_report_end trace point 
Patch series "Add error_report_end tracepoint to KFENCE and KASAN", v3.

This patchset adds a tracepoint, error_repor_end, that is to be used by
KFENCE, KASAN, and potentially other bug detection tools, when they print
an error report.  One of the possible use cases is userspace collection of
kernel error reports: interested parties can subscribe to the tracing
event via tracefs, and get notified when an error report occurs.

This patch (of 3):

Introduce error_report_end tracepoint.  It can be used in debugging tools
like KASAN, KFENCE, etc.  to provide extensions to the error reporting
mechanisms (e.g.  allow tests hook into error reporting, ease error report
collection from production kernels).  Another benefit would be making use
of ftrace for debugging or benchmarking the tools themselves.

Should we need it, the tracepoint name leaves us with the possibility to
introduce a complementary error_report_start tracepoint in the future.

Link: https://lkml.kernel.org/r/20210121131915.1331302-1-glider@google.com
Link: https://lkml.kernel.org/r/20210121131915.1331302-2-glider@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Suggested-by: Marco Elver <elver@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit ba7612c00686f204f7bca4ceb7394a9e705e84bd
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ic86e29982c04dad4b3b7889a424f37b22cc5f22b
 2026-04-28 18:54:34 +00:00
Marco Elver cc59eafba6 FROMGIT: kfence: show access type in report 
Show the access type in KFENCE reports by plumbing through read/write
information from the page fault handler.  Update the documentation and
test accordingly.

Link: https://lkml.kernel.org/r/20210111091544.3287013-2-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Suggested-by: Jörn Engel <joern@purestorage.com>
Reviewed-by: Jörn Engel <joern@purestorage.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit e29117c1fbf30d27d5afe41cf34263e1fd8e4f04
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I2e9bb224292cf92ac828232c51cd57024ac56d7d
 2026-04-28 18:54:34 +00:00
Marco Elver cf4d20195e FROMGIT: kfence: fix typo in test 
Fix a typo/accidental copy-paste that resulted in the obviously
incorrect 'GFP_KERNEL * 2' expression.

Link: https://lkml.kernel.org/r/X9lHQExmHGvETxY4@elver.google.com
Signed-off-by: Marco Elver <elver@google.com>
Reported-by: kernel test robot <lkp@intel.com>
Acked-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 161c8770c371ca3992565d8f1db1ad4a88a562e9
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I4c26617c64fd2e6d410e6e793bb0eebf8fc87e55
 2026-04-28 18:54:34 +00:00
Marco Elver 79d23f598b BACKPORT: kfence: add test suite 
Add KFENCE test suite, testing various error detection scenarios. Makes
use of KUnit for test organization. Since KFENCE's interface to obtain
error reports is via the console, the test verifies that KFENCE outputs
expected reports to the console.

Link: https://lkml.kernel.org/r/20201103175841.3495947-9-elver@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit d6364119849bb0432e9a46e9699519ea9ff1bb77
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: removed memcg sanity checks from the tests]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I733090d4109a795c078fe8090c46b19cdfe9413f
 2026-04-28 18:54:34 +00:00
Marco Elver 90c3900ad6 FROMGIT: kfence: add missing copyright header to documentation 
Add missing copyright header to KFENCE documentation.

Link: https://lkml.kernel.org/r/20210118092159.145934-4-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 14c8d0148b7b44c549c8892a93a800a9650fa31c
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iaf953c9bf1f220054b1ec4068f2ba0bd9a7e9fc4
 2026-04-28 18:54:34 +00:00
Marco Elver 1817d6edcf BACKPORT: kfence, Documentation: add KFENCE documentation 
Add KFENCE documentation in dev-tools/kfence.rst, and add to index.

Link: https://lkml.kernel.org/r/20201103175841.3495947-8-elver@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit f63eeaffb09b57d4c9cb9c92c4925ee3bd3df457
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: fixed a minor conflict in Documentation/dev-tools/index.rst]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iaedafbf20944d68c87229f6c8b2a6e5155fcee60
 2026-04-28 18:54:34 +00:00
Alexander Potapenko 6f3edbf1cb BACKPORT: kfence, kasan: make KFENCE compatible with KASAN 
Make KFENCE compatible with KASAN. Currently this helps test KFENCE
itself, where KASAN can catch potential corruptions to KFENCE state, or
other corruptions that may be a result of freepointer corruptions in the
main allocators.

Link: https://lkml.kernel.org/r/20201103175841.3495947-7-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Co-developed-by: Marco Elver <elver@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 8ab944ae627dc9fb165bff68acc465751a0b8de2
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: resolved a conflict in mm/kasan/generic.c]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I2f862c2e514e7fcff50a019048c8f0d22f46e6c4
 2026-04-28 18:54:34 +00:00
Alexander Potapenko c8cbb13941 BACKPORT: mm, kfence: insert KFENCE hooks for SLUB 
Inserts KFENCE hooks into the SLUB allocator.

To pass the originally requested size to KFENCE, add an argument
'orig_size' to slab_alloc*(). The additional argument is required to
preserve the requested original size for kmalloc() allocations, which
uses size classes (e.g. an allocation of 272 bytes will return an object
of size 512). Therefore, kmem_cache::size does not represent the
kmalloc-caller's requested size, and we must introduce the argument
'orig_size' to propagate the originally requested size to KFENCE.

Without the originally requested size, we would not be able to detect
out-of-bounds accesses for objects placed at the end of a KFENCE object
page if that object is not equal to the kmalloc-size class it was
bucketed into.

When KFENCE is disabled, there is no additional overhead, since
slab_alloc*() functions are __always_inline.

Link: https://lkml.kernel.org/r/20201103175841.3495947-6-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Co-developed-by: Marco Elver <elver@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 5aa4d4f541f500cd63f814ae39bea55024ed5c6b
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: dropped changes to include/linux/slub_def.h,
         fixed conflict in mm/slub.c,
         always pass root cache pointer to kfence_alloc()]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iacfc22088087cb920107a595f71b09e21d5f2f47
 2026-04-28 18:54:34 +00:00
Alexander Potapenko 2fe50ef221 BACKPORT: mm, kfence: insert KFENCE hooks for SLAB 
Inserts KFENCE hooks into the SLAB allocator.

To pass the originally requested size to KFENCE, add an argument
'orig_size' to slab_alloc*(). The additional argument is required to
preserve the requested original size for kmalloc() allocations, which
uses size classes (e.g. an allocation of 272 bytes will return an object
of size 512). Therefore, kmem_cache::size does not represent the
kmalloc-caller's requested size, and we must introduce the argument
'orig_size' to propagate the originally requested size to KFENCE.

Without the originally requested size, we would not be able to detect
out-of-bounds accesses for objects placed at the end of a KFENCE object
page if that object is not equal to the kmalloc-size class it was
bucketed into.

When KFENCE is disabled, there is no additional overhead, since
slab_alloc*() functions are __always_inline.

Link: https://lkml.kernel.org/r/20201103175841.3495947-5-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Marco Elver <elver@google.com>

Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jann Horn <jannh@google.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 840c0553e89413319d67971a321bcc07114da9b8
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: resolved minor API change in mm/slab_common.c,
         dropped changes to include/linux/slab_def.h,
         resolved conflicts in mm/slab.c,
         always pass root cache pointer to kfence_alloc]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iab2ba9c7b06b9a234d93ba892be639941861f8ab
 2026-04-28 18:54:34 +00:00
Marco Elver aa44dfbaab UPSTREAM: mm/kasan: add object validation in ksize() 
ksize() has been unconditionally unpoisoning the whole shadow memory
region associated with an allocation.  This can lead to various undetected
bugs, for example, double-kzfree().

Specifically, kzfree() uses ksize() to determine the actual allocation
size, and subsequently zeroes the memory.  Since ksize() used to just
unpoison the whole shadow memory region, no invalid free was detected.

This patch addresses this as follows:

1. Add a check in ksize(), and only then unpoison the memory region.

2. Preserve kasan_unpoison_slab() semantics by explicitly unpoisoning
   the shadow memory region using the size obtained from __ksize().

Tested:
1. With SLAB allocator: a) normal boot without warnings; b) verified the
   added double-kzfree() is detected.
2. With SLUB allocator: a) normal boot without warnings; b) verified the
   added double-kzfree() is detected.

[elver@google.com: s/BUG_ON/WARN_ON_ONCE/, per Kees]
  Link: http://lkml.kernel.org/r/20190627094445.216365-6-elver@google.com
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=199359
Link: http://lkml.kernel.org/r/20190626142014.141844-6-elver@google.com
Change-Id: I74fb5904ad52c6f5555d86af33b40050c8faf5d2
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Marco Elver afaa642262 UPSTREAM: mm/kasan: change kasan_check_{read,write} to return boolean 
This changes {,__}kasan_check_{read,write} functions to return a boolean
denoting if the access was valid or not.

[sfr@canb.auug.org.au: include types.h for "bool"]
  Link: http://lkml.kernel.org/r/20190705184949.13cdd021@canb.auug.org.au
Link: http://lkml.kernel.org/r/20190626142014.141844-3-elver@google.com
Change-Id: Ib725b6caf26995121bccedf8561cbe43094569be
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Kees Cook <keescook@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Marco Elver 8e3089d59b UPSTREAM: mm/kasan: introduce __kasan_check_{read,write} 
Patch series "mm/kasan: Add object validation in ksize()", v3.

This patch (of 5):

This introduces __kasan_check_{read,write}.  __kasan_check functions may
be used from anywhere, even compilation units that disable instrumentation
selectively.

This change eliminates the need for the __KASAN_INTERNAL definition.

[elver@google.com: v5]
  Link: http://lkml.kernel.org/r/20190708170706.174189-2-elver@google.com
Link: http://lkml.kernel.org/r/20190626142014.141844-2-elver@google.com
Change-Id: Iddfae00044e339498cd71fcddb0bec2b6345b907
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Marco Elver 0e1432322b UPSTREAM: mm/slab: refactor common ksize KASAN logic into slab_common.c 
This refactors common code of ksize() between the various allocators into
slab_common.c: __ksize() is the allocator-specific implementation without
instrumentation, whereas ksize() includes the required KASAN logic.

Link: http://lkml.kernel.org/r/20190626142014.141844-5-elver@google.com
Change-Id: Ie1ed4a70559f9bbd395a1c0774a02776d7bc48f6
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Christoph Lameter <cl@linux.com>
Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Kees Cook <keescook@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Kees Cook bdcefa1e51 UPSTREAM: mm/slab: sanity-check page type when looking up cache 
This avoids any possible type confusion when looking up an object.  For
example, if a non-slab were to be passed to kfree(), the invalid
slab_cache pointer (i.e.  overlapped with some other value from the
struct page union) would be used for subsequent slab manipulations that
could lead to further memory corruption.

Since the page is already in cache, adding the PageSlab() check will
have nearly zero cost, so add a check and WARN() to virt_to_cache().
Additionally replaces an open-coded virt_to_cache().  To support the
failure mode this also updates all callers of virt_to_cache() and
cache_from_obj() to handle a NULL cache pointer return value (though
note that several already handle this case gracefully).

[dan.carpenter@oracle.com: restore IRQs in kfree()]
  Link: http://lkml.kernel.org/r/20190613065637.GE16334@mwanda
Link: http://lkml.kernel.org/r/20190530045017.15252-3-keescook@chromium.org
Change-Id: I7c98ef20b38af63ff58dc80fb5655e8be40cd617
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Cc: Alexander Popov <alex.popov@linux.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Kees Cook 54280aefda UPSTREAM: mm/slab: validate cache membership under freelist hardening 
Patch series "mm/slab: Improved sanity checking".

This adds defenses against slab cache confusion (as seen in real-world
exploits[1]) and gracefully handles type confusions when trying to look
up slab caches from an arbitrary page.  (Also is patch 3: new LKDTM
tests for these defenses as well as for the existing double-free
detection.

This patch (of 3):

When building under CONFIG_SLAB_FREELIST_HARDENING, it makes sense to
perform sanity-checking on the assumed slab cache during
kmem_cache_free() to make sure the kernel doesn't mix freelists across
slab caches and corrupt memory (as seen in the exploitation of flaws
like CVE-2018-9568[1]).  Note that the prior code might WARN() but still
corrupt memory (i.e.  return the assumed cache instead of the owned
cache).

There is no noticeable performance impact (changes are within noise).
Measuring parallel kernel builds, I saw the following with
CONFIG_SLAB_FREELIST_HARDENED, before and after this patch:

before:

	Run times: 288.85 286.53 287.09 287.07 287.21
	Min: 286.53 Max: 288.85 Mean: 287.35 Std Dev: 0.79

after:

	Run times: 289.58 287.40 286.97 287.20 287.01
	Min: 286.97 Max: 289.58 Mean: 287.63 Std Dev: 0.99

Delta: 0.1% which is well below the standard deviation

[1] https://github.com/ThomasKing2014/slides/raw/master/Building%20universal%20Android%20rooting%20with%20a%20type%20confusion%20vulnerability.pdf

Link: http://lkml.kernel.org/r/20190530045017.15252-2-keescook@chromium.org
Change-Id: Ibb7e2ef88efb2fb9cd30fa8707a2bc0a759c2ede
Signed-off-by: Kees Cook <keescook@chromium.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Alexander Popov <alex.popov@linux.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 2026-04-28 18:54:34 +00:00
Alexander Popov dcc3522b15 FROMGIT: mm/slab: rerform init_on_free earlier 
Currently in CONFIG_SLAB init_on_free happens too late, and heap objects
go to the heap quarantine not being erased.

Lets move init_on_free clearing before calling kasan_slab_free().  In that
case heap quarantine will store erased objects, similarly to CONFIG_SLUB=y
behavior.

Link: https://lkml.kernel.org/r/20201210183729.1261524-1-alex.popov@linux.com
Signed-off-by: Alexander Popov <alex.popov@linux.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Bug: 177201466
(cherry picked from commit a32d654db543843a5ffb248feaec1a909718addd
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I2bf5e70c1524619526efd792bbdd959b813af1e4
 2026-04-28 18:54:34 +00:00
Marco Elver 66fb355247 FROMGIT: kfence: use pt_regs to generate stack trace on faults 
Instead of removing the fault handling portion of the stack trace based on
the fault handler's name, just use struct pt_regs directly.

Change kfence_handle_page_fault() to take a struct pt_regs, and plumb it
through to kfence_report_error() for out-of-bounds, use-after-free, or
invalid access errors, where pt_regs is used to generate the stack trace.

If the kernel is a DEBUG_KERNEL, also show registers for more information.

Link: https://lkml.kernel.org/r/20201105092133.2075331-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Suggested-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 54a5abe9b5d542ee71836439cc662efe178c8211
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I3a60060b24f0efb4faee2e6c953973bc1263e8d1
 2026-04-28 18:54:34 +00:00
Marco Elver 399a3b5b04 FROMGIT: kfence, arm64: add missing copyright and description header 
Add missing copyright and description header to KFENCE source file.

Link: https://lkml.kernel.org/r/20210118092159.145934-3-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit ce19707fadfe62b7441f2a9455b6866161fbc1f0
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I75c06e57e4bb906626c13318d00cfc7d47b76bde
 2026-04-28 18:54:34 +00:00
Marco Elver 2c94402fda BACKPORT: arm64, kfence: enable KFENCE for ARM64 
Add architecture specific implementation details for KFENCE and enable
KFENCE for the arm64 architecture. In particular, this implements the
required interface in <asm/kfence.h>.

KFENCE requires that attributes for pages from its memory pool can
individually be set. Therefore, force the entire linear map to be mapped
at page granularity. Doing so may result in extra memory allocated for
page tables in case rodata=full is not set; however, currently
CONFIG_RODATA_FULL_DEFAULT_ENABLED=y is the default, and the common case
is therefore not affected by this change.

Link: https://lkml.kernel.org/r/20201103175841.3495947-4-elver@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit d3cb0555da9b469c3f9ebe663d4c0d6265757175
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: resolved a minor conflict in arch/arm64/Kconfig]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ibe9d26c2c9088862ba8855c18e0c5210a68c7f50
 2026-04-28 18:54:34 +00:00
Marco Elver 280fcbae34 FROMGIT: kfence, x86: add missing copyright and description header 
Add missing copyright and description header to KFENCE source file.

Link: https://lkml.kernel.org/r/20210118092159.145934-2-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 8470d53956b7e1b6f3f2dd17f277b9d2bb1472ae
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I90aef6f80e7b5a46f5168d1a24c02a5737785831
 2026-04-28 18:54:34 +00:00
Alexander Potapenko 6853b2999b BACKPORT: x86, kfence: enable KFENCE for x86 
Add architecture specific implementation details for KFENCE and enable
KFENCE for the x86 architecture. In particular, this implements the
required interface in <asm/kfence.h> for setting up the pool and
providing helper functions for protecting and unprotecting pages.

For x86, we need to ensure that the pool uses 4K pages, which is done
using the set_memory_4k() helper function.

Link: https://lkml.kernel.org/r/20201103175841.3495947-3-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Marco Elver <elver@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit cb99fcd83140d0d58ea36db6c1c2034abc95f983
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: resolved a minor conflict in arch/x86/Kconfig,
         s/flush_tlb_one_kernel/__flush_tlb_one_kernel]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I111caffb0b88c34ed9ff57b95f127b08eacedcb9
 2026-04-28 18:54:34 +00:00
Marco Elver f9e10d18fe FROMGIT: kfence: add missing copyright and description headers 
Add missing copyright and description headers to KFENCE source files.

Link: https://lkml.kernel.org/r/20210118092159.145934-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit b86d1ed1155ce1d2420057bfbdcc62b9fd53c1d6
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I95eb6756baaa0d8ca1dbc656708667372cff546d
 2026-04-28 18:54:34 +00:00
Marco Elver ebc5b341bb FROMGIT: kfence: add option to use KFENCE without static keys 
For certain usecases, specifically where the sample interval is always
set to a very low value such as 1ms, it can make sense to use a dynamic
branch instead of static branches due to the overhead of toggling a
static branch.

Therefore, add a new Kconfig option to remove the static branches and
instead check kfence_allocation_gate if a KFENCE allocation should be
set up.

Link: https://lkml.kernel.org/r/20210111091544.3287013-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Suggested-by: Jörn Engel <joern@purestorage.com>
Reviewed-by: Jörn Engel <joern@purestorage.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit c01761611b325c1e4ec7d3e236cc9db003cb82fd
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I68a112a8ff68fa24742b198e036f130a9757c27f
 2026-04-28 18:54:34 +00:00
Marco Elver f0699a2c5c FROMGIT: kfence: fix potential deadlock due to wake_up() 
Lockdep reports that we may deadlock when calling wake_up() in
__kfence_alloc(), because we may already hold base->lock.  This can happen
if debug objects are enabled:

    ...
    __kfence_alloc+0xa0/0xbc0 mm/kfence/core.c:710
    kfence_alloc include/linux/kfence.h:108 [inline]
    ...
    kmem_cache_zalloc include/linux/slab.h:672 [inline]
    fill_pool+0x264/0x5c0 lib/debugobjects.c:171
    __debug_object_init+0x7a/0xd10 lib/debugobjects.c:560
    debug_object_init lib/debugobjects.c:615 [inline]
    debug_object_activate+0x32c/0x3e0 lib/debugobjects.c:701
    debug_timer_activate kernel/time/timer.c:727 [inline]
    __mod_timer+0x77d/0xe30 kernel/time/timer.c:1048
    ...

Therefore, switch to an open-coded wait loop.  The difference to before is
that the waiter wakes up and rechecks the condition after 1 jiffy;
however, given the infrequency of kfence allocations, the difference is
insignificant.

Link: https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com
Link: https://lkml.kernel.org/r/20210104130749.1768991-1-elver@google.com
Reported-by: syzbot+8983d6d4f7df556be565@syzkaller.appspotmail.com
Signed-off-by: Marco Elver <elver@google.com>
Suggested-by: Hillf Danton <hdanton@sina.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit c5fb1ab1a3c6d0ee02d1054a10d51ffcac57aed5
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iee40e9f216afbc3fce8e43c0e2a4bc807fdddf39
 2026-04-28 18:54:34 +00:00
Marco Elver c862f6124f FROMGIT: kfence: avoid stalling work queue task without allocations 
To toggle the allocation gates, we set up a delayed work that calls
toggle_allocation_gate().  Here we use wait_event() to await an allocation
and subsequently disable the static branch again.  However, if the kernel
has stopped doing allocations entirely, we'd wait indefinitely, and stall
the worker task.  This may also result in the appropriate warnings if
CONFIG_DETECT_HUNG_TASK=y.

Therefore, introduce a 1 second timeout and use wait_event_timeout().  If
the timeout is reached, the static branch is disabled and a new delayed
work is scheduled to try setting up an allocation at a later time.

Note that, this scenario is very unlikely during normal workloads once the
kernel has booted and user space tasks are running.  It can, however,
happen during early boot after KFENCE has been enabled, when e.g.  running
tests that do not result in any allocations.

Link: https://lkml.kernel.org/r/CADYN=9J0DQhizAGB0-jz4HOBBh+05kMBXb4c0cXMS7Qi5NAJiw@mail.gmail.com
Link: https://lkml.kernel.org/r/20201110135320.3309507-1-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Reported-by: Anders Roxell <anders.roxell@linaro.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 80d4693491f6f20de01437319b081fdda2079e67
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I2332ff8144b8bce5c4574b01ea2863e0e71e6124
 2026-04-28 18:54:34 +00:00
Marco Elver 86809000d8 FROMGIT: kfence: Fix parameter description for kfence_object_start() 
Describe parameter @addr correctly by delimiting with ':'.

Link: https://lkml.kernel.org/r/20201106092149.GA2851373@elver.google.com
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 65f5b471bfd099a54862e14a895724e982a381c9
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ieb71f57f82351eaaabd3c63cd52e97fbfbfca2f1
 2026-04-28 18:54:34 +00:00
Alexander Potapenko f64ca19413 BACKPORT: mm: add Kernel Electric-Fence infrastructure 
Patch series "KFENCE: A low-overhead sampling-based memory safety error detector", v7.

This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a
low-overhead sampling-based memory safety error detector of heap
use-after-free, invalid-free, and out-of-bounds access errors.  This
series enables KFENCE for the x86 and arm64 architectures, and adds
KFENCE hooks to the SLAB and SLUB allocators.

KFENCE is designed to be enabled in production kernels, and has near
zero performance overhead. Compared to KASAN, KFENCE trades performance
for precision. The main motivation behind KFENCE's design, is that with
enough total uptime KFENCE will detect bugs in code paths not typically
exercised by non-production test workloads. One way to quickly achieve a
large enough total uptime is when the tool is deployed across a large
fleet of machines.

KFENCE objects each reside on a dedicated page, at either the left or
right page boundaries. The pages to the left and right of the object
page are "guard pages", whose attributes are changed to a protected
state, and cause page faults on any attempted access to them. Such page
faults are then intercepted by KFENCE, which handles the fault
gracefully by reporting a memory access error.

Guarded allocations are set up based on a sample interval (can be set
via kfence.sample_interval). After expiration of the sample interval,
the next allocation through the main allocator (SLAB or SLUB) returns a
guarded allocation from the KFENCE object pool. At this point, the timer
is reset, and the next allocation is set up after the expiration of the
interval.

To enable/disable a KFENCE allocation through the main allocator's
fast-path without overhead, KFENCE relies on static branches via the
static keys infrastructure. The static branch is toggled to redirect the
allocation to KFENCE.

The KFENCE memory pool is of fixed size, and if the pool is exhausted no
further KFENCE allocations occur. The default config is conservative
with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB
pages).

We have verified by running synthetic benchmarks (sysbench I/O,
hackbench) and production server-workload benchmarks that a kernel with
KFENCE (using sample intervals 100-500ms) is performance-neutral
compared to a non-KFENCE baseline kernel.

KFENCE is inspired by GWP-ASan [1], a userspace tool with similar
properties. The name "KFENCE" is a homage to the Electric Fence Malloc
Debugger [2].

For more details, see Documentation/dev-tools/kfence.rst added in the
series -- also viewable here:

	https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst

[1] http://llvm.org/docs/GwpAsan.html
[2] https://linux.die.net/man/3/efence

This patch (of 9):

This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a
low-overhead sampling-based memory safety error detector of heap
use-after-free, invalid-free, and out-of-bounds access errors.

KFENCE is designed to be enabled in production kernels, and has near
zero performance overhead. Compared to KASAN, KFENCE trades performance
for precision. The main motivation behind KFENCE's design, is that with
enough total uptime KFENCE will detect bugs in code paths not typically
exercised by non-production test workloads. One way to quickly achieve a
large enough total uptime is when the tool is deployed across a large
fleet of machines.

KFENCE objects each reside on a dedicated page, at either the left or
right page boundaries. The pages to the left and right of the object
page are "guard pages", whose attributes are changed to a protected
state, and cause page faults on any attempted access to them. Such page
faults are then intercepted by KFENCE, which handles the fault
gracefully by reporting a memory access error. To detect out-of-bounds
writes to memory within the object's page itself, KFENCE also uses
pattern-based redzones. The following figure illustrates the page
layout:

  ---+-----------+-----------+-----------+-----------+-----------+---
     | xxxxxxxxx | O :       | xxxxxxxxx |       : O | xxxxxxxxx |
     | xxxxxxxxx | B :       | xxxxxxxxx |       : B | xxxxxxxxx |
     | x GUARD x | J : RED-  | x GUARD x | RED-  : J | x GUARD x |
     | xxxxxxxxx | E :  ZONE | xxxxxxxxx |  ZONE : E | xxxxxxxxx |
     | xxxxxxxxx | C :       | xxxxxxxxx |       : C | xxxxxxxxx |
     | xxxxxxxxx | T :       | xxxxxxxxx |       : T | xxxxxxxxx |
  ---+-----------+-----------+-----------+-----------+-----------+---

Guarded allocations are set up based on a sample interval (can be set
via kfence.sample_interval). After expiration of the sample interval, a
guarded allocation from the KFENCE object pool is returned to the main
allocator (SLAB or SLUB). At this point, the timer is reset, and the
next allocation is set up after the expiration of the interval.

To enable/disable a KFENCE allocation through the main allocator's
fast-path without overhead, KFENCE relies on static branches via the
static keys infrastructure. The static branch is toggled to redirect the
allocation to KFENCE. To date, we have verified by running synthetic
benchmarks (sysbench I/O, hackbench) that a kernel compiled with KFENCE
is performance-neutral compared to the non-KFENCE baseline.

For more details, see Documentation/dev-tools/kfence.rst (added later in
the series).

Link: https://lkml.kernel.org/r/20201103175841.3495947-2-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: SeongJae Park <sjpark@amazon.de>
Co-developed-by: Marco Elver <elver@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Bug: 177201466
(cherry picked from commit 2a8dede73c3496bbd917644657f3735a4f508cb9
    https://github.com/hnaz/linux-mm v5.11-rc4-mmots-2021-01-21-20-10)
[glider: dropped KCSAN bits missing in 5.4,
         resolved minor conflict in init/main.c]
Test: CONFIG_KFENCE_KUNIT_TEST=y passes on Cuttlefish
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I6b474675cc9732c31118df53fa06c3997f577218
 2026-04-28 18:54:34 +00:00
Thomas Turner 9d54f9ffdb fixup! UPSTREAM: netlink: make validation more configurable for future strictness 
For KVER < 4.12.0 we use nla_parse

Change-Id: I1940b6a570ae9fd85844d807db74c7a07a63354d
 2026-04-25 15:54:46 +01:00
Thomas Turner b0da80ea68 Merge tag 'v4.19.325-cip131' of https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip into android13-4.19-kona 
version 4.19.325-cip131

* tag 'v4.19.325-cip131' of https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip:
  CIP: Bump version suffix to -cip131 after merge from cip/linux-4.19.y-st tree
  Update localversion-st, tree is up-to-date with 5.10.252.
  nfsd: fix return error code for nfsd_map_name_to_[ug]id
  net: usb: pegasus: enable basic endpoint checking
  Bluetooth: L2CAP: Fix not checking l2cap_chan security level
  Bluetooth: Enforce key size of 16 bytes on FIPS level
  xfrm6: fix uninitialized saddr in xfrm6_get_saddr()
  netfilter: nf_conntrack: Add allow_clash to generic protocol handler
  ext4: don't cache extent during splitting extent
  NTB: ntb_transport: Fix too small buffer for debugfs_name
  net: ethernet: ec_bhf: Fix dma_free_coherent() dma handle
  fbdev: ffb: fix corrupted video output on Sun FFB1
  fbdev: of: display_timing: fix refcount leak in of_get_display_timings()
  atm: fore200e: fix use-after-free in tasklets during device removal
  net: wan/fsl_ucc_hdlc: Fix dma_free_coherent() in uhdlc_memclean()
  net: ethernet: marvell: skge: remove incorrect conflicting PCI ID
  ceph: supply snapshot context in ceph_zero_partial_object()
  MIPS: rb532: Fix MMIO UART resource registration
  parisc: kernel: replace kfree() with put_device() in create_tree_node()
  drm/amdkfd: Fix out-of-bounds write in kfd_event_page_set()
  usb: dwc2: fix resume failure if dr_mode is host
  btrfs: continue trimming remaining devices on failure
  kexec: derive purgatory entry from symbol
  ocfs2: fix reflink preserve cleanup issue
  ocfs2: fix xattr array entry __counted_by error
  rapidio: replace rio_free_net() with kfree() in rio_scan_alloc_net()
  iio: gyro: itg3200: Fix unchecked return value in read_raw
  dm mpath: make pg_init_delay_msecs settable
  bus: fsl-mc: fix an error handling in fsl_mc_device_add()
  md/bitmap: fix GPF in write_page caused by resize race
  xfs: fix remote xattr valuelblk check
  xfs: fix freemap adjustments when adding xattrs to leaf blocks
  xfs: delete attr leaf freemap entries when empty
  xfs: mark data structures corrupt on EIO and ENODATA
  mm, page_alloc, thp: prevent reclaim for __GFP_THISNODE THP allocations
  drm: of: drm_of_panel_bridge_remove(): fix device_node leak
  dm-unstripe: fix mapping bug when there are multiple targets in a table
  clk: tegra: tegra124-emc: Fix potential memory leak in tegra124_clk_register_emc()
  dm: clear cloned request bio pointer when last clone bio completes
  media: i2c/tw9906: Fix potential memory leak in tw9906_probe()
  media: i2c/tw9903: Fix potential memory leak in tw9903_probe()
  media: cx25821: Add missing unmap in snd_cx25821_hw_params()
  media: cx23885: Add missing unmap in snd_cx23885_hw_params()
  media: cx88: Add missing unmap in snd_cx88_hw_params()
  media: radio-keene: fix memory leak in error path
  HID: prodikeys: Check presence of pm->input_ep82
  HID: magicmouse: Do not crash on missing msc->input
  HID: hid-pl: handle probe errors
  media: mtk-mdp: Fix a reference leak bug in mtk_mdp_remove()
  dm-verity: correctly handle dm_bufio_client_create() failure
  fpga: dfl: use subsys_initcall to allow built-in drivers to be added
  rpmsg: core: fix race in driver_override_show() and use core helper
  netfilter: nf_conntrack_h323: fix OOB read in decode_choice()
  net: consume xmit errors of GSO frames
  Bluetooth: L2CAP: Fix missing key size check for L2CAP_LE_CONN_REQ
  Bluetooth: l2cap: Check encryption key size on incoming connection
  wifi: cfg80211: wext: fix IGTK key ID off-by-one
  net: usb: lan78xx: scan all MDIO addresses on LAN7801
  lan78xx: Fix memory allocation bug
  lan78xx: Fix race condition in disconnect handling
  lan78xx: Remove unused pause frame queue
  net: usb: kaweth: remove TX queue manipulation in kaweth_set_rx_mode
  ntb: ntb_hw_switchtec: Fix shift-out-of-bounds for 0 mw lut
  ntb: ntb_hw_switchtec: Fix array-index-out-of-bounds access
  drm/radeon: Add HAINAN clock adjustment
  drm/amdgpu: Add HAINAN clock adjustment
  ARM: 9467/1: mm: Don't use %pK through printk
  include: uapi: netfilter_bridge.h: Cover for musl libc
  iio: magnetometer: Remove IRQF_ONESHOT
  Revert "mfd: da9052-spi: Change read-mask to write-mask"
  binder: don't use %pK through printk
  serial: 8250_dw: handle clock enable errors in runtime_resume
  m68k: nommu: fix memmove() with differently aligned src and dest for 68000
  clk: microchip: core: correct return value on *_get_parent()
  nfc: nxp-nci: remove interrupt trigger type
  myri10ge: avoid uninitialized variable use
  PCI: Mark Nvidia GB10 to avoid bus reset
  PCI: Fix pci_slot_lock () device locking
  PCI: Mark ASM1164 SATA controller to avoid bus reset
  net/rds: Clear reconnect pending bit
  vmw_vsock: bypass false-positive Wnonnull warning with gcc-16
  net: usb: sr9700: remove code to drive nonexistent multicast filter
  Bluetooth: btusb: Add device ID for Realtek RTL8761BU
  Bluetooth: hci_conn: use mod_delayed_work for active mode timeout
  ipv4: fib: Annotate access to struct fib_alias.fa_state.
  wifi: iwlegacy: add missing mutex protection in il3945_store_measurement()
  wifi: iwlegacy: add missing mutex protection in il4965_store_tx_power()
  net/rds: No shortcut out of RDS_CONN_ERROR
  net: usb: r8152: fix transmit queue timeout
  openrisc: define arch-specific version of nop()
  netfilter: xt_tcpmss: check remaining length before reading optlen
  wifi: libertas: fix WARNING in usb_tx_block
  jfs: nlink overflow in jfs_rename
  jfs: Add missing set_freezable() for freezable kthread
  modpost: Amend ppc64 save/restfpr symnames for -Os build
  ASoC: es8328: Add error unwind in resume
  HID: multitouch: add eGalaxTouch EXC3188 support
  drm/atmel-hlcdc: fix use-after-free of drm_crtc_commit after release
  drm/atmel-hlcdc: fix memory leak from the atomic_destroy_state callback
  virt: vbox: uapi: Mark inner unions in packed structs as packed
  hyper-v: Mark inner union in hv_kvp_exchg_msg_value as packed
  drm: Account property blob allocations to memcg
  media: cx25821: Fix a resource leak in cx25821_dev_setup()
  media: solo6x10: Check for out of bounds chip_id
  media: pvrusb2: fix URB leak in pvr2_send_request_ex
  media: adv7180: fix frame interval in progressive mode
  ASoC: wm8962: Add WM8962_ADC_MONOMIX to "3D Coefficients" mask
  media: omap3isp: set initial format
  media: omap3isp: isppreview: always clamp in preview_try_format()
  media: omap3isp: isp_video_mbus_to_pix/pix_to_mbus fixes
  media: dvb-core: dmxdevfilter must always flush bufs
  parisc: Prevent interrupts during reboot
  arm64: tegra: smaug: Add usb-role-switch support
  pstore: ram_core: fix incorrect success return when vmap() fails
  mailbox: bcm-ferxrm-mailbox: Use default primary handler
  clocksource/drivers/sh_tmu: Always leave device running after probe
  sparc: don't reference obsolete termio struct for TC* constants
  xenbus: Use .freeze/.thaw to handle xenbus devices
  ACPICA: Abort AML bytecode execution when executing AML_FATAL_OP
  EFI/CPER: don't go past the ARM processor CPER record buffer
  x86/xen/pvh: Enable PAE mode for 32-bit guest only when CONFIG_X86_PAE is set
  s390/purgatory: Add -Wno-default-const-init-unsafe to KBUILD_CFLAGS
  tools/power cpupower: Reset errno before strtoull()
  minix: Add required sanity checking to minix_check_superblock()
  hfsplus: pretend special inodes as regular files
  audit: add missing syscalls to read class
  hfsplus: fix volume corruption issue for generic/498
  audit: add fchmodat2() to change attributes class
  rtc: interface: Alarm race handling should not discard preceding error
  SUNRPC: fix gss_auth kref leak in gss_alloc_msg error path
  SUNRPC: auth_gss: fix memory leaks in XDR decoding error paths
  ata: pata_ftide010: Fix some DMA timings
  ext4: fix memory leak in ext4_ext_shift_extents()
  MIPS: Work around LLVM bug when gp is used as global register variable
  apparmor: fix invalid deref of rawdata when export_binary is unset
  apparmor: fix rlimit for posix cpu timers
  apparmor: fix NULL sock in aa_sock_file_perm
  bonding: alb: fix UAF in rlb_arp_recv during bond up/down
  macvlan: observe an RCU grace period in macvlan_common_newlink() error path
  netfilter: nf_conntrack_h323: don't pass uninitialised l3num value
  net/rds: rds_sendmsg should not discard payload_len
  xen-netback: reject zero-queue configuration from guest
  net: usb: catc: enable basic endpoint checking
  usbb: catc: use correct API for MAC addresses
  cpuidle: Skip governor when only one idle state is available
  pinctrl: single: fix refcount leak in pcs_add_gpio_func()
  iio: sca3000: Fix a resource leak in sca3000_probe()
  ovl: Fix uninit-value in ovl_fill_real
  drivers: iio: mpu3050: use dev_err_probe for regulator request
  mfd: arizona: Fix regulator resource leak on wm5102_clear_write_sequencer() failure
  Revert "mmc: rtsx_pci_sdmmc: increase power-on settling delay to 5ms"
  mmc: rtsx_pci_sdmmc: increase power-on settling delay to 5ms
  mmc: core: Initial support for SD express card/host
  serial: SH_SCI: improve "DMA support" prompt
  staging: greybus: lights: avoid NULL deref
  fbdev: au1200fb: Fix a memory leak in au1200fb_drv_probe()
  tracing: Remove duplicate ENABLE_EVENT_STR and DISABLE_EVENT_STR macros
  scsi: csiostor: Fix dereference of null pointer rn
  pNFS: fix a missing wake up while waiting on NFS_LAYOUT_DRAIN
  svcrdma: Remove queue-shortening warnings
  RDMA/core: Fix a couple of obvious typos in comments
  power: supply: wm97xx_battery: Convert to GPIO descriptor
  power: supply: bq27xxx: fix wrong errno when bus ops are unsupported
  power: supply: sbs-battery: Fix use-after-free in power_supply_changed()
  power: supply: rt9455: Fix use-after-free in power_supply_changed()
  power: supply: act8945a: Fix use-after-free in power_supply_changed()
  serial: caif: fix use-after-free in caif_serial ldisc_close()
  net: atm: fix crash due to unvalidated vcc pointer in sigd_send()
  procfs: fix missing RCU protection when reading real_parent in do_task_stat()
  netfilter: nft_set_rbtree: check for partial overlaps in anonymous sets
  netfilter: nft_set_hash: fix get operation on big endian
  bonding: only set speed/duplex to unknown, if getting speed failed
  ucount: check for CAP_SYS_RESOURCE using ns_capable_noaudit()
  tcp: tcp_tx_timestamp() must look at the rtx queue
  fat: avoid parent link count underflow in rmdir
  PCI: Mark 3ware-9650SA Root Port Extended Tags as broken
  netfilter: nf_conncount: fix tracking of connections from localhost
  netfilter: nf_conncount: increase the connection clean up limit to 64
  netfilter: nf_conncount: make nf_conncount_gc_list() to disable BH
  wifi: cfg80211: stop NAN and P2P in cfg80211_leave
  PCI/portdrv: Fix potential resource leak
  PCI: Do not attempt to set ExtTag for VFs
  PCI: mediatek: Fix IRQ domain leak when MSI allocation fails
  Revert "hwmon: (ibmpex) fix use-after-free in high/low store"
  spi: tools: Add include folder to .gitignore
  regulator: Flag uncontrollable regulators as always_on
  regulator: core: Respect off_on_delay at startup
  smack: /smack/doi: accept previously used values
  smack: /smack/doi must be > 0
  arm64: dts: amlogic: gx: assign the MMC signal clocks
  ARM: dts: allwinner: sun5i-a13-utoo-p66: delete "power-gpios" property
  EDAC/i5400: Fix snprintf() limit calculation in calculate_dimm_size()
  EDAC/i5000: Fix snprintf() size calculation in calculate_dimm_size()
  pstore/ram: fix buffer overflow in persistent_ram_save_old()
  sched/rt: Skip currently executing CPU in rto_next_cpu()
  mfd: wm8350-core: Use IRQF_ONESHOT
  EDAC/altera: Remove IRQF_ONESHOT
  hrtimer: Fix trace oddity
  crypto: cavium - fix dma_free_coherent() size
  s390/cio: Fix device lifecycle handling in css_alloc_subchannel()
  PM: sleep: wakeirq: harden dev_pm_clear_wake_irq() against races
  md/raid10: fix any_working flag handling in raid10_sync_request
  tpm: st33zp24: Fix missing cleanup on get_burstcount() error
  tpm: tpm_i2c_infineon: Fix locality leak on get_burstcount() failure
  gfs2: Add metapath_dibh helper
  gfs2: Add wrapper for iomap_file_buffered_write
  fs: add <linux/init_task.h> for 'init_fs'
  hfsplus: return error when node already exists in hfs_bnode_create
  auxdisplay: arm-charlcd: fix release_mem_region() size
  RDMA/umad: Reject negative data_len in ib_umad_write

Change-Id: Ice1efa8b2faaaab14d1d1563e5aa6ae36549682b
 2026-04-22 07:08:43 +01:00
Bush-cat e6bff35ad8 fixup! BACKPORT: cgroup: make per-cgroup pressure stall tracking configurable 
Commit f7c2472acb had a conflict in kernel/cgroup/cgroup.c
Remove the duplicated loop in cgroup_disable.

Change-Id: I6d830e3397c020723750221495f7a0825de5f5c3
 2026-04-21 19:58:17 +00:00
Tashfin Shakeer Rhythm 9d058b5eba Reapply "mm: make faultaround produce old ptes" 
On Android, application launch latency and direct reclaim time are reduced by
10%+ and ~80% respectively [1] and this repository is targeted at Android. The
revert was done because of a ~6% regression in unibench on x86, not Android.

Do not handicap Android just to fix a x86 regression which won't be used at all
on ARM64.

This reverts commit 9775c770cd.

[1] https://github.com/torvalds/linux/commit/46bdb4277f98e70d0c91f4289897ade533fe9e80

Change-Id: I8e0cfb8733d56213ab481ed5e6711533d1d89b9f
Signed-off-by: Tashfin Shakeer Rhythm <tashfinshakeerrhythm@gmail.com>
 2026-04-21 04:58:22 +00:00
Kyle Lin b7e673b219 arm64: configs: vendor: Switch linker to LLD 
If we enable lld, we also enable the CONFIG_LTO_CLANG to avoid the build
breakage. We need to do two things
1. To enable CONFIG_LTO_CLANG in defconfig
2. Clean the "-d LTO" and "-d LTO_CLANG" from build configs

THINLTO also disabled and we enable it when using *.config.no-cfi to
reduce the build time.

Bug: 140525226
Change-Id: I7adb9f9e474acddad078447be17777355d884d20
Signed-off-by: Kyle Lin <kylelin@google.com>
 2026-04-20 23:15:57 -04:00
Jason Chiu 62c1b32987 arm64: configs: vendor: Enable CONFIG_SHADOW_CALL_STACK 
Bug: 147125983
Bug: 147262842
Change-Id: I9ca8df7f299d02e73cb5471551b67a71693896bd
Signed-off-by: Jason Chiu <jasoncschiu@google.com>
 2026-04-20 23:15:57 -04:00
Alistair Delva 658b7f2434 arm64: configs: vendor: Enable CONFIG_RD_LZ4 
Support future decompression of LZ4-compressed ramdisk images.

This support is in addition to the default support for gzip.

Bug: 150391496
Test: TreeHugger
Change-Id: I8c07e9363333c53cd6ff9c7e16fc42cbe318fe83
Signed-off-by: Alistair Delva <adelva@google.com>
 2026-04-20 23:15:57 -04:00
Sami Tolvanen 769464ab10 msm: ipa3: fix callback function type for ndo_start_xmit 
Bug: 148470764
Bug: 117299373
Change-Id: Iad7ed493bd29e2da85e2d6bc0f1dc0fbdac8a41d
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
 2026-04-20 23:15:57 -04:00
Thomas Turner 871df7a3f1 thermal: qcom: fix KCFI errors 
Change-Id: Iee0d6fee40aeadfd169421bc0de265f9c1726766
 2026-04-20 23:15:57 -04:00
Flicker372 43054a6fa3 msm: vidc/cvp: fix function type for hfi_cmd_response_callback 
Change-Id: I5ae2a546fa08343a7e16dc6aff8183c08a5f3ea2
Signed-off-by: Flicker372 flicker372@outlook.com
 2026-04-20 23:15:57 -04:00
Miguel de Dios 68084f3505 msm: vidc/cvp: Fix handle_cmd_response parameter type 
Bug: 117299373
Change-Id: Iffee5fd84fb1340bed7585d3dfb438e7955f61c5
Signed-off-by: Miguel de Dios <migueldedios@google.com>
Signed-off-by: Flicker372 <flicker372@outlook.com>
 2026-04-20 23:15:56 -04:00