bf3d16fa44f57a0446c0b22c6ce0434a55441a08
822089
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marco Ballesio
|
bf3d16fa44 |
FROMGIT: binder: BINDER_GET_FROZEN_INFO ioctl
PROPAGATED_from (CR) User space needs to know if binder transactions occurred to frozen processes. Introduce a new BINDER_GET_FROZEN ioctl and keep track of transactions occurring to frozen proceses. Bug: 180989544 (cherry picked from commit c55019c24b22d6770bd8e2f12fbddf3f83d37547 git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git char-misc-testing) Signed-off-by: Marco Ballesio <balejs@google.com> Signed-off-by: Li Li <dualli@google.com> Acked-by: Todd Kjos <tkjos@google.com> Link: https://lore.kernel.org/r/20210316011630.1121213-4-dualli@chromium.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Change-Id: Ifb760a32f016fdfedb816319a5c2fa2e9dddc5ff Reviewed-on: https://gerrit.mot.com/2119046 Reviewed-by: Vijayakumar Gn <vijaygn@motorola.com> Reviewed-by: Ling Jin <lingjin@motorola.com> SLTApproved: Slta Waiver SME-Granted: SME Approvals Granted Tested-by: Jira Key Submit-Approved: Jira Key Signed-off-by: Christian Bernard Gioia <chrisb@motorola.com> Reviewed-on: https://gerrit.mot.com/2130306 Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> |
||
|
marco Ballesio
|
cc240be298 |
binder: BINDER_FREEZE ioctl
PROPAGATED_from (CR) Frozen tasks can't process binder transactions, so a way is required to inform transmitting ends of communication failures due to the frozen state of their receiving counterparts. Additionally, races are possible between transitions to frozen state and binder transactions enqueued to a specific process. Implement BINDER_FREEZE ioctl for user space to inform the binder driver about the intention to freeze or unfreeze a process. When the ioctl is called, block the caller until any pending binder transactions toward the target process are flushed. Return an error to transactions to processes marked as frozen. Change-Id: Iccd8c52a3f35207b2bc294bd5dc6bcca4dda4437 Signed-off-by: Marco Ballesio <balejs@google.com> Co-developed-by: Todd Kjos <tkjos@google.com> Signed-off-by: Todd Kjos <tkjos@google.com> Signed-off-by: Li Li <dualli@google.com> Signed-off-by: Kanagarathina kalathi <kalath(CR)@motorola.com> Reviewed-on: https://gerrit.mot.com/2108850 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Srihari Sathyanarayana <sriharis@motorola.com> Reviewed-by: Vijayakumar Gn <vijaygn@motorola.com> Reviewed-by: Ling Jin <lingjin@motorola.com> Submit-Approved: Jira Key Signed-off-by: Christian Bernard Gioia <chrisb@motorola.com> Reviewed-on: https://gerrit.mot.com/2122168 Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> |
||
|
Jann Horn
|
4f7076e434 |
mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse
commit 2555283eb40df89945557273121e9393ef9b542b upstream.
anon_vma->degree tracks the combined number of child anon_vmas and VMAs
that use the anon_vma as their ->anon_vma.
anon_vma_clone() then assumes that for any anon_vma attached to
src->anon_vma_chain other than src->anon_vma, it is impossible for it to
be a leaf node of the VMA tree, meaning that for such VMAs ->degree is
elevated by 1 because of a child anon_vma, meaning that if ->degree
equals 1 there are no VMAs that use the anon_vma as their ->anon_vma.
This assumption is wrong because the ->degree optimization leads to leaf
nodes being abandoned on anon_vma_clone() - an existing anon_vma is
reused and no new parent-child relationship is created. So it is
possible to reuse an anon_vma for one VMA while it is still tied to
another VMA.
This is an issue because is_mergeable_anon_vma() and its callers assume
that if two VMAs have the same ->anon_vma, the list of anon_vmas
attached to the VMAs is guaranteed to be the same. When this assumption
is violated, vma_merge() can merge pages into a VMA that is not attached
to the corresponding anon_vma, leading to dangling page->mapping
pointers that will be dereferenced during rmap walks.
Fix it by separately tracking the number of child anon_vmas and the
number of VMAs using the anon_vma as their ->anon_vma.
Mot-CRs-fixed: (CR)
CVE-Fixed: CVE-2022-42703
Bug: 253167854
Fixes:
|
||
|
Gilberto Gambugge
|
657b260037 |
techpack: add camera legacy support
Add camera legacy support by checking BOARD_CONFIG_CAMERA_KERNEL_NAME to decide which folder will be used to compile the camera techpack. Also, due to camera legacy, the camera uapi kernel headers installation routines need to be updated to consider the camera legacy support. This commit also fixes the routines to install kernel headers for 'make headers_install' and also 'qti_generate_kernel_headers_arm' and 'qti_generate_kernel_headers_arm64'. Change-Id: I845c5cb49c388bc15eba2fbf0ed0724e93de5192 Reviewed-on: https://gerrit.mot.com/2632695 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Reviewed-by: Rafael Ortolan <rafones@motorola.com> Reviewed-by: Rav(CR) Vembu <raviv@motorola.com> Reviewed-by: Miguel Nicodemus <miguelh@motorola.com> Tested-by: Jira Key Submit-Approved: Jira Key |
||
|
Lee Jones
|
73fa1b6be0 |
ANDROID: usb: f_accessory: Check buffer size when initialised via composite
When communicating with accessory devices via USBFS, the initialisation call-stack looks l(CR): ConfigFS > Gadget ConfigFS > UDC > Gadget ConfigFS > Composite Eventually ending up in composite_dev_prepare() where memory for the data buffer is allocated and initialised. The default size used for the allocation is USB_COMP_EP0_BUFSIZ (4k). When handling bulk transfers, acc_ctrlrequest() needs to be able to handle buffers up to BULK_BUFFER_SIZE (16k). Instead of adding new generic attributes to struct usb_request to track the size of the allocated buffer, we can simply split off the affected thread of execution to travel via a knowledgeable abstracted function acc_ctrlrequest_composite() where we can complete the necessary specific checks. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2023-2094 Bug: 264029575 Signed-off-by: Lee Jones <joneslee@google.com> Change-Id: I38542bc1f09b79df9a1800d218a8229814d43e46 Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2556743 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Clement Lecigne
|
b5030682a5 |
ALSA: pcm: Move rwsem lock inside snd_ctl_elem_read to prevent UAF
[ Note: this is a fix that works around the bug equivalently as the
two upstream commits:
1fa4445f9adf ("ALSA: control - introduce snd_ctl_notify_one() helper")
56b88b50565c ("ALSA: pcm: Move rwsem lock inside snd_ctl_elem_read to prevent UAF")
but in a simpler way to fit with older stable trees -- tiwai ]
Add missing locking in ctl_elem_read_user/ctl_elem_write_user which can be
easily triggered and turned into an use-after-free.
Example code paths with SNDRV_CTL_IOCTL_ELEM_READ:
64-bits:
snd_ctl_ioctl
snd_ctl_elem_read_user
[takes controls_rwsem]
snd_ctl_elem_read [lock properly held, all good]
[drops controls_rwsem]
32-bits (compat):
snd_ctl_ioctl_compat
snd_ctl_elem_write_read_compat
ctl_elem_write_read
snd_ctl_elem_read [missing lock, not good]
CVE-2023-0266 was assigned for this issue.
Mot-CRs-fixed: (CR)
CVE-Fixed: CVE-2023-0266
Bug: 265303544
Change-Id: I2584396230b1f87531ea4819cb82785082174f8b
Signed-off-by: Clement Lecigne <clecigne@google.com>
Cc: stable@kernel.org # 5.12 and older
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Reviewed-by: Jaroslav Kysela <perex@perex.cz>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com>
Reviewed-on: https://gerrit.mot.com/2530929
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Hyunwoo Kim
|
5a59256232 |
video: fbdev: pxa3xx-gcu: Fix integer overflow in pxa3xx_gcu_write
[ Upstream commit a09d2d00af53b43c6f11e6ab3cb58443c2cac8a7 ] In pxa3xx_gcu_write, a count parameter of type size_t is passed to words of type int. Then, copy_from_user() may cause a heap overflow because it is used as the third argument of copy_from_user(). Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-39842 Bug: 245928838 Change-Id: If3b27ac3720242550e7e9b5547590085a75ee5ba Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com> Signed-off-by: Helge Deller <deller@gmx.de> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2480183 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Konstantin Makariev <kmakariev@motorola.com> Submit-Approved: Jira Key |
||
|
Anirudh Rayabharam
|
158859083d |
ext4: fix kernel infoleak via ext4_extent_header
commit ce3aba43599f0b50adbebff133df8d08a3d5fffe upstream.
Initialize eh_generation of struct ext4_extent_header to prevent leaking
info to userspace. Fixes KMSAN kernel-infoleak bug reported by syzbot at:
http://syzkaller.appspot.com/bug?id=78e9ad0e6952a3ca16e8234724b2fa92d041b9b8
Mot-CRs-fixed: (CR)
CVE-Fixed: CVE-2022-0850
Bug: 245406696
Change-Id: I7136669bc3fa829a870111dcfb5ebb1914149531
Cc: stable@kernel.org
Reported-by: syzbot+2dcfeaf8cb49b05e8f1a@syzkaller.appspotmail.com
Fixes:
|
||
|
Johannes Berg
|
f3834a92dd |
wifi: mac80211_hwsim: avoid mac80211 warning on bad rate
commit 1833b6f46d7e2830251a063935ab464256defe22 upstream. If the tool on the other side (e.g. wmediumd) gets confused about the rate, we hit a warning in mac80211. Silence that by effectively duplicating the check here and dropping the frame silently (in mac80211 it's dropped with the warning). Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-41674, CVE-2022-42719, CVE-2022-42720, CVE-2022-42721 Bug: 253641805, 253642087, 253642015, 253642088 Change-Id: Ic01c4edc808d23b41ab9a1c3c2d99e0135dc8209 Reported-by: Sönke Huster <shuster@seemoo.tu-darmstadt.de> Tested-by: Sönke Huster <shuster@seemoo.tu-darmstadt.de> Signed-off-by: Johannes Berg <johannes.berg@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2468493 SLTApproved: Slta Waiver SME-Granted: SME Approvals Granted Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Helge Deller
|
fa6d036bca |
fbcon: Disallow setting font bigger than screen size
commit 65a01e601dbba8b7a51a2677811f70f783766682 upstream. Prevent that users set a font size which is bigger than the physical screen. It's unl(CR) this may happen (because screens are usually much larger than the fonts and each font char is limited to 32x32 pixels), but it may happen on smaller screens/LCD displays. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2021-33655 Bug: 240019719 Change-Id: I9830f341a6cf1cd2f65c5dc8e2ef4b7bf3a62092 Signed-off-by: Helge Deller <deller@gmx.de> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Reviewed-by: Geert Uytterhoeven <geert@linux-m68k.org> Cc: stable@vger.kernel.org # v4.14+ Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2458683 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
James Morse
|
bcc86e2aae |
KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated
commit a5905d6af492ee6a4a2205f0d550b3f931b03d03 upstream. KVM allows the guest to discover whether the ARCH_WORKAROUND SMCCC are implemented, and to preserve that state during migration through its firmware register interface. Add the necessary boiler plate for SMCCC_ARCH_WORKAROUND_3. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-23960 Bug: 215557547 Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> [ kvm code moved to virt/kvm/arm, removed fw regs ABI. Added 32bit stub ] Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Change-Id: I4bade20f2192f41f3455360d625aa3f39d1d18ef Reviewed-on: https://gerrit.mot.com/2359726 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Anshuman Khandual
|
c699d83f54 |
arm64: Add Cortex-X2 CPU part definition
commit 72bb9dcb6c33cfac80282713c2b4f2b254cd24d1 upstream. Add the CPU Partnumbers for the new Arm designs. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-23960 Bug: 215557547 Change-Id: I8cbfa7f738c1318f27c5b380e6ba6c27701480d8 Cc: Will Deacon <will@kernel.org> Cc: Suzuki Poulose <suzuki.poulose@arm.com> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Link: https://lore.kernel.org/r/1642994138-25887-2-git-send-email-anshuman.khandual@arm.com Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2359723 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Suzuki K Poulose
|
f70297bbb0 |
arm64: Add Neoverse-N2, Cortex-A710 CPU part definition
commit 2d0d656700d67239a57afaf617439143d8dac9be upstream. Add the CPU Partnumbers for the new Arm designs. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-23960 Bug: 215557547 Change-Id: Ic8c1d59199af04e8192dd57e89b046154acca0cb Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Will Deacon <will@kernel.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> Link: https://lore.kernel.org/r/20211019163153.3692640-2-suzuki.poulose@arm.com Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2359722 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Rob Herring
|
66ea0c0f00 |
arm64: Add part number for Arm Cortex-A77
commit 8a6b88e66233f5f1779b0a1342aa9dc030dddcd5 upstream. Add the MIDR part number info for the Arm Cortex-A77. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2022-23960 Bug: 215557547 Change-Id: If2d9be37235146dce569642f08c7a3e11fdaa047 Signed-off-by: Rob Herring <robh@kernel.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20201028182839.166037-1-robh@kernel.org Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2359721 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Linus Torvalds
|
db6226f89c |
fget: check that the fd still exists after getting a ref to it
commit 054aa8d439b9185d4f5eb9a90282d1ce74772969 upstream.
Jann Horn points out that there is another possible race wrt Unix domain
socket garbage collection, somewhat reminiscent of the one fixed in
commit cbcf01128d0a ("af_unix: fix garbage collect vs MSG_PEEK").
See the extended comment about the garbage collection requirements added
to unix_peek_fds() by that commit for details.
The race comes from how we can locklessly look up a file descriptor just
as it is in the process of being closed, and with the right artificial
timing (Jann added a few strategic 'mdelay(500)' calls to do that), the
Unix domain socket garbage collector could see the reference count
decrement of the close() happen before fget() took its reference to the
file and the file was attached onto a new file descriptor.
This is all (intentionally) correct on the 'struct file *' side, with
RCU lookups and lockless reference counting very much part of the
design. Getting that reference count out of order isn't a problem per
se.
But the garbage collector can get confused by seeing this situation of
having seen a file not having any remaining external references and then
seeing it being attached to an fd.
In commit cbcf01128d0a ("af_unix: fix garbage collect vs MSG_PEEK") the
fix was to serialize the file descriptor install with the garbage
collector by taking and releasing the unix_gc_lock.
That's not really an option here, but since this all happens when we are
in the process of looking up a file descriptor, we can instead simply
just re-check that the file hasn't been closed in the meantime, and just
re-do the lookup if we raced with a concurrent close() of the same file
descriptor.
Mot-CRs-fixed: (CR)
CVE-Fixed: CVE-2021-4083
Bug: 216408350
Change-Id: I8a7ec2ad919f3036a88bd0288c5a8177a4ded57c
Reported-and-tested-by: Jann Horn <jannh@google.com>
Acked-by: M(CR) Szeredi <mszeredi@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com>
Reviewed-on: https://gerrit.mot.com/2370933
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Huosheng Liao <liaohs@motorola.com>
Submit-Approved: Jira Key
|
||
|
Jens Axboe
|
dd1145ae4d |
fs: add fget_many() and fput_many()
commit 091141a42e15fe47ada737f3996b317072afcefb upstream. Some uses cases repeatedly get and put references to the same file, but the only exposed interface is doing these one at the time. As each of these entail an atomic inc or dec on a shared structure, that cost can add up. Add fget_many(), which works just l(CR) fget(), except it takes an argument for how many references to get on the file. Ditto fput_many(), which can drop an arbitrary number of references to a file. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2021-4083 Bug: 216408350 Change-Id: I56d79c45029900342bbdff74d3a4686255813d07 Reviewed-by: Hannes Reinecke <hare@suse.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2370932 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Huosheng Liao <liaohs@motorola.com> Submit-Approved: Jira Key |
||
|
Linus Torvalds
|
0ef4646bd1 |
mm: fix misplaced unlock_page in do_wp_page()
Commit 09854ba94c6a ("mm: do_wp_page() simplification") reorganized all
the code around the page re-use vs copy, but in the process also moved
the final unlock_page() around to after the wp_page_reuse() call.
That normally doesn't matter - but it means that the unlock_page() is
now done after releasing the page table lock. Again, not a big deal,
you'd think.
But it turns out that it's very wrong indeed, because once we've
released the page table lock, we've basically lost our only reference to
the page - the page tables - and it could now be free'd at any time. We
do hold the mmap_sem, so no actual unmap() can happen, but madvise can
come in and a MADV_DONTNEED will zap the page range - and free the page.
So now the page may be free'd just as we're unlocking it, which in turn
will usually trigger a "Bad page state" error in the freeing path. To
make matters more confusing, by the time the debug code prints out the
page state, the unlock has typically completed and everything looks fine
again.
This all doesn't happen in any normal situations, but it does trigger
with the dirtyc0w_child LTP test. And it seems to trigger much more
easily (but not expclusively) on s390 than elsewhere, probably because
s390 doesn't do the "batch pages up for freeing after the TLB flush"
that gives the unlock_page() more time to complete and makes the race
harder to hit.
Mot-CRs-fixed: (CR)
CVE-Fixed: CVE-2020-29374
Bug: 174737879
Fixes: 09854ba94c6a ("mm: do_wp_page() simplification")
Link: https://lore.kernel.org/lkml/a46e9bbef2ed4e17778f5615e818526ef848d791.camel@redhat.com/
Link: https://lore.kernel.org/linux-mm/c41149a8-211e-390b-af1d-d5eee690fecb@linux.alibaba.com/
Reported-by: Qian Cai <cai@redhat.com>
Reported-by: Alex Shi <alex.shi@linux.alibaba.com>
Bisected-and-analyzed-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Tested-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Git-commit: be068f29034fb00530a053d18b8cf140c32b12b3
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
[zhenhuah@codeaurora.org: resolved conflict]
Change-Id: I775e3706aa5d6be6b8352e8f59cb10e8d9326c5c
Signed-off-by: Zhenhua Huang <zhenhuah@codeaurora.org>
Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com>
Reviewed-on: https://gerrit.mot.com/2313292
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Konstantin Makariev <kmakariev@motorola.com>
Submit-Approved: Jira Key
|
||
|
Linus Torvalds
|
ab87f621f7 |
mm: do_wp_page() simplification
How about we just make sure we're the only possible valid user fo the page before we bother to reuse it? Simplify, simplify, simplify. And get rid of the nasty serialization on the page lock at the same time. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2020-29374 Bug: 174737879 Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> [peterx: add subject prefix] Signed-off-by: Peter Xu <peterx@redhat.com> Git-commit: 09854ba94c6aad7886996bfbee2530b3d8a7f4f4 Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git [zhenhuah@codeaurora.org: resolve trivial merge conflicts] Change-Id: I73dc102cb02d751d497bdf3cc39179a4bfdc5a54 Signed-off-by: Zhenhua Huang <zhenhuah@codeaurora.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2313290 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Konstantin Makariev <kmakariev@motorola.com> Submit-Approved: Jira Key |
||
|
Peter Xu
|
4a77274788 |
mm/ksm: Remove reuse_ksm_page()
Remove the function as the last reference has gone away with the do_wp_page() changes. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2020-29374 Bug: 174737879 Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Git-commit: 1a0cf26323c80e2f1c58fc04f15686de61bfab0c Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git Change-Id: Icd11ae747c4b0b9ddd81b8b3c5e463ed88abea9c Signed-off-by: Zhenhua Huang <zhenhuah@codeaurora.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Reviewed-on: https://gerrit.mot.com/2313291 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Konstantin Makariev <kmakariev@motorola.com> Submit-Approved: Jira Key |
||
|
Kirill Tkhai
|
9d2b9e1615 |
mm: reuse only-pte-mapped KSM page in do_wp_page()
Add an optimization for KSM pages almost in the same way that we have for ordinary anonymous pages. If there is a write fault in a page, which is mapped to an only pte, and it is not related to swap cache; the page may be reused without copying its content. [ Note that we do not consider PageSwapCache() pages at least for now, since we don't want to complicate __get_ksm_page(), which has nice optimization based on this (for the migration case). Currenly it is spinning on PageSwapCache() pages, waiting for when they have unfreezed counters (i.e., for the migration finish). But we don't want to make it also spinning on swap cache pages, which we try to reuse, since there is not a very high probability to reuse them. So, for now we do not consider PageSwapCache() pages at all. ] So in reuse_ksm_page() we check for 1) PageSwapCache() and 2) page_stable_node(), to skip a page, which KSM is currently trying to link to stable tree. Then we do page_ref_freeze() to prohibit KSM to merge one more page into the page, we are reusing. After that, nobody can refer to the reusing page: KSM skips !PageSwapCache() pages with zero refcount; and the protection against of all other participants is the same as for reused ordinary anon pages pte lock, page lock and mmap_sem. Mot-CRs-fixed: (CR) CVE-Fixed: CVE-2020-29374 Bug: 174737879 [akpm@linux-foundation.org: replace BUG_ON()s with WARN_ON()s] Link: http://lkml.kernel.org/r/154471491016.31352.1168978849911555609.stgit@localhost.localdomain Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com> Reviewed-by: Yang Shi <yang.shi@linux.alibaba.com> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Hugh Dickins <hughd@google.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Christian Koenig <christian.koenig@amd.com> Cc: Claudio Imbrenda <imbrenda@linux.vnet.ibm.com> Cc: R(CR) van Riel <riel@surriel.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Minchan Kim <minchan@kernel.org> Cc: Kirill Tkhai <ktkhai@virtuozzo.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Gajjala Chakradhar <gajjalac@motorola.com> Change-Id: I503e824fd8d389be74ba768a6e1c6143318e2b33 Reviewed-on: https://gerrit.mot.com/2313289 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Konstantin Makariev <kmakariev@motorola.com> Submit-Approved: Jira Key |
||
|
a17671
|
8eff5221e1 |
Revert "mm:Fix conflict of speculative fault and MGLRU"
Per Zhaoyu suggestion This reverts commit 644eec929177d354ceed5ae60b2d34defcf8b24d. So the LMK kill times could be dropped with the MGLRU feature Change-Id: I63d2398c2e694cb5687de592b35d9ff45af23720 Moto-CRs-Fixed:(CR) Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2323140 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
edb74fec61 |
mm:vmscan:Update the page scan prioritiy
Update the page scan priority For qcom 4.19 kernel baseline Moto-CRs-Fixed:(CR) Change-Id: Ic6026b9512c971d6cb8832a2a78fef7586a12453 Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2323141 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Bruno Oliveira
|
30b094f614 |
skip USB PHY reset on charger removal
WHAT: this commit disables the reset of Qualcomm usb phy when the charger is removed WHY: at caprip this is putting the PMIC at a undefined state, with black screen and unresponsive kernel Change-Id: I7fb035f81760f91b94f2d61778002e1975a4032a Reviewed-on: https://gerrit.mot.com/2312756 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Reviewed-by: Rafael Ortolan <rafones@motorola.com> Reviewed-by: Gilberto Gambugge Neto <gambugge@motorola.com> Reviewed-by: Marcos Sassahara <msassaha@motorola.com> Tested-by: Jira Key Submit-Approved: Jira Key |
||
|
a17671
|
e90e7499c7 |
fuse:Porting fuse passthrough feature
Android common kernel already support fuse passthrough Backport to qcom sm2020 baseline Change-Id: Icd1791f5a4cc07a8af080426f69124764e0872a2 Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2283027 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
dc9d750ffe |
mm: mglru fix on qcom baseline
Rebase the 4.19 mglru patches on qcom baseline Add vm events for more information Moto-CRs-Fixed:(CR) Change-Id: I81bfef4c09340ff0564cb37de915a2ce3fd0526d Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2281592 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
a17671
|
875fb38be8 |
mm:Clean page_io revert conflict
Moto zram writeback has conflicts with mglru There is some merge error after reverted the zram writeback changes, clean it for correct logic Change-Id: I4ba5093dae7eab5ecc3cb16bab0ddfe793342306 Reviewed-on: https://gerrit.mot.com/2281591 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
wanggl3
|
62642ba9ca |
Revert write back changes
(CR) mm : zram fix swapcached issue on Zram Writeback Change-Id: I66df9637f31a206edb8fbb10876984ccb53e1890 Reviewed-on: https://gerrit.mot.com/2271684 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Wang Wang <wangwang1@lenovo.com> Reviewed-by: Zhangqing Huang <huangzq2@motorola.com> Submit-Approved: Jira Key |
||
|
xieqiong2
|
84acf0f38d |
iio:add bc12 type read iio.
add bc12 type read iio seperately Change-Id: Id5f3b8952f1ef8fb1dac85682bd09346b9c4c0b5 Signed-off-by: xieqiong2 <xieqiong2@lenovo.com> Reviewed-on: https://gerrit.mot.com/2273506 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Wei Xu <xuwei9@lenovo.com> Reviewed-by: Wang Wang <wangwang1@lenovo.com> Submit-Approved: Jira Key |
||
|
hqjiangy
|
621a819638 |
[rhode]:Modify ldo driver voltage[2/2]
Change-Id: Ic793c60df69005a6a695abc3167797e77fb73dce Signed-off-by: hqjiangy <hqjiangy@motorola.com> Reviewed-on: https://gerrit.mot.com/2269516 SLTApproved: Slta Waiver SME-Granted: SME Approvals Granted Reviewed-by: Hua Tan <tanhua1@motorola.com> Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Tested-by: Jira Key Submit-Approved: Jira Key |
||
|
a17671
|
86c55e9c07 |
configs:Enable multi generation LRU
Enable multi-generation LRU feature For better OOM performance Change-Id: I90a54ec90b641e98e8eddda4d6681d0a92ff3f90 Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2269556 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Will Deacon
|
6813d8e85b |
arm64: pgtable: Fix pte_accessible()
pte_accessible() is used by ptep_clear_flush() to figure out whether TLB
invalidation is necessary when unmapping pages for reclaim. Although our
implementation is correct according to the architecture, returning true
only for valid, young ptes in the absence of racing page-table
modifications, this is in fact flawed due to lazy invalidation of old
ptes in ptep_clear_flush_young() where we elide the expensive DSB
instruction for completing the TLB invalidation.
Rather than penalise the aging path, adjust pte_accessible() to return
true for any valid pte, even if the access flag is cleared.
Moto-CRs-Fixed:(CR)
Cc: <stable@vger.kernel.org>
Fixes:
|
||
|
Minchan Kim
|
733d86b3a2 |
mm: fix unexpected zeroed page mapping with zram swap
Two processes under CLONE_VM cloning, user process can be corrupted by
seeing zeroed page unexpectedly.
CPU A CPU B
do_swap_page do_swap_page
SWP_SYNCHRONOUS_IO path SWP_SYNCHRONOUS_IO path
swap_readpage valid data
swap_slot_free_notify
delete zram entry
swap_readpage zeroed(invalid) data
pte_lock
map the *zero data* to userspace
pte_unlock
pte_lock
if (!pte_same)
goto out_nomap;
pte_unlock
return and next refault will
read zeroed data
The swap_slot_free_notify is bogus for CLONE_VM case since it doesn't
increase the refcount of swap slot at copy_mm so it couldn't catch up
whether it's safe or not to discard data from backing device. In the
case, only the lock it could rely on to synchronize swap slot freeing is
page table lock. Thus, this patch gets rid of the swap_slot_free_notify
function. With this patch, CPU A will see correct data.
CPU A CPU B
do_swap_page do_swap_page
SWP_SYNCHRONOUS_IO path SWP_SYNCHRONOUS_IO path
swap_readpage original data
pte_lock
map the original data
swap_free
swap_range_free
bd_disk->fops->swap_slot_free_notify
swap_readpage read zeroed data
pte_unlock
pte_lock
if (!pte_same)
goto out_nomap;
pte_unlock
return
on next refault will see mapped data by CPU B
The concern of the patch would increase memory consumption since it
could keep wasted memory with compressed form in zram as well as
uncompressed form in address space. However, most of cases of zram uses
no readahead and do_swap_page is followed by swap_free so it will free
the compressed form from in zram quickly.
Moto-CRs-Fixed:(CR)
Link: https://lkml.kernel.org/r/YjTVVxIAsnKAXjTd@google.com
Fixes:
|
||
|
Yu Zhao
|
d0c00b17a0 |
mm:Fix conflict of speculative fault and MGLRU
the LRU algorithm doesn't apply to sequential or random reads handle this case in speculative fault Moto-CRs-Fixed:(CR) Change-Id: Ia968c3903e05ee0acb6ae503537ece47e80b099f Signed-off-by: a17671 <a17671@motorola.com> Reviewed-on: https://gerrit.mot.com/2270849 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
d3d2daf25e |
FROMLIST: mm: multi-gen LRU: design doc
Add a design doc. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-15-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: I1d66302e618416291ebf9647e20625fb76613c89 Reviewed-on: https://gerrit.mot.com/2268706 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
a764ce0e0e |
FROMLIST: mm: multi-gen LRU: admin guide
Add an admin guide. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-14-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: I6fafbd7eb3ef6819cfcd30376459f14893f17c63 Reviewed-on: https://gerrit.mot.com/2268705 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
0ed1bd23b9 |
FROMLIST: mm: multi-gen LRU: debugfs interface
Add /sys/kernel/debug/lru_gen for working set estimation and proactive reclaim. These features are required to optimize job scheduling (bin packing) in data centers [1][2]. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages: 1. It offers better choices because it is aware of memcgs, NUMA nodes, shared mappings and unmapped page cache. 2. It is more scalable because it is O(nr_hot_pages), whereas the PFN-based approach is O(nr_total_pages). Add /sys/kernel/debug/lru_gen_full for debugging. [1] https://dl.acm.org/doi/10.1145/3297858.3304053 [2] https://dl.acm.org/doi/10.1145/3503222.3507731 Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-13-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: Ie558098e0a24a647f77f4eacc4d72576173fc0b8 Reviewed-on: https://gerrit.mot.com/2268704 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
73bbccfe48 |
FROMLIST: mm: multi-gen LRU: thrashing prevention
Add /sys/kernel/mm/lru_gen/min_ttl_ms for thrashing prevention, as requested by many desktop users [1]. When set to value N, it prevents the working set of N milliseconds from getting evicted. The OOM killer is triggered if this working set cannot be kept in memory. Based on the average human detectable lag (~100ms), N=1000 usually eliminates intolerable lags due to thrashing. Larger values l(CR) N=3000 make lags less noticeable at the risk of premature OOM kills. Compared with the size-based approach, e.g., [2], this time-based approach has the following advantages: 1. It is easier to configure because it is agnostic to applications and memory sizes. 2. It is more reliable because it is directly wired to the OOM killer. [1] https://lore.kernel.org/lkml/Ydza%2FzXKY9ATRoh6@google.com/ [2] https://lore.kernel.org/lkml/20211130201652.2218636d@mail.inbox.lv/ Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-12-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: I482d33f3beaf7723d2f3eeaaa5b4f12bcb9b48a1 Reviewed-on: https://gerrit.mot.com/2268703 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
a9ef9de5f8 |
FROMLIST: mm: multi-gen LRU: kill switch
Add /sys/kernel/mm/lru_gen/enabled as a kill switch. Components that
can be disabled include:
0x0001: the multi-gen LRU core
0x0002: walking page table, when arch_has_hw_pte_young() returns
true
0x0004: clearing the accessed bit in non-leaf PMD entries, when
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y
[yYnN]: apply to all the components above
E.g.,
echo y >/sys/kernel/mm/lru_gen/enabled
cat /sys/kernel/mm/lru_gen/enabled
0x0007
echo 5 >/sys/kernel/mm/lru_gen/enabled
cat /sys/kernel/mm/lru_gen/enabled
0x0005
NB: the page table walks happen on the scale of seconds under heavy
memory pressure, in which case the mmap_lock contention is a lesser
concern, compared with the LRU lock contention and the I/O congestion.
So far the only well-known case of the mmap_lock contention happens on
Android, due to Scudo [1] which allocates several thousand VMAs for
merely a few hundred MBs. The SPF and the Maple Tree also have
provided their own assessments [2][3]. However, if walking page tables
does worsen the mmap_lock contention, the kill switch can be used to
disable it. In this case the multi-gen LRU will suffer a minor
performance degradation, as shown previously.
Clearing the accessed bit in non-leaf PMD entries can also be
disabled, since this behavior was not tested on x86 varieties other
than Intel and AMD.
Moto-CRs-Fixed:(CR)
[1] https://source.android.com/devices/tech/debug/scudo
[2] https://lore.kernel.org/lkml/20220128131006.67712-1-michel@lespinasse.org/
[3] https://lore.kernel.org/lkml/20220202024137.2516438-1-Liam.Howlett@oracle.com/
Link: https://lore.kernel.org/r/20220309021230.721028-11-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I71801d9470a2588cad8bfd14fbcfafc7b010aa03
Reviewed-on: https://gerrit.mot.com/2268702
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
834be483c2 |
FROMLIST: mm: multi-gen LRU: optimize multiple memcgs
When multiple memcgs are available, it is possible to make better
choices based on generations and tiers and therefore improve the
overall performance under global memory pressure. This patch adds a
rudimentary optimization to select memcgs that can drop single-use
unmapped clean pages first. Doing so reduces the chance of going into
the aging path or swapping. These two operations can be costly.
A typical example that benefits from this optimization is a server
running mixed types of workloads, e.g., heavy anon workload in one
memcg and heavy buffered I/O workload in the other.
Though this optimization can be applied to both kswapd and direct
reclaim, it is only added to kswapd to keep the patchset manageable.
Later improvements will cover the direct reclaim path.
Server benchmark results:
Mixed workloads:
fio (buffered I/O): -[23, 25]%
IOPS BW
patch1-8: 2960k 11.3GiB/s
patch1-9: 2248k 8783MiB/s
memcached (anon): +[210, 214]%
Ops/sec KB/sec
patch1-8: 606940.09 23576.89
patch1-9: 1895197.49 73619.93
Mixed workloads:
fio (buffered I/O): -[4, 6]%
IOPS BW
5.18-ed4643521e6a: 2369k 9255MiB/s
patch1-9: 2248k 8783MiB/s
memcached (anon): +[510, 516]%
Ops/sec KB/sec
5.18-ed4643521e6a: 309189.58 12010.61
patch1-9: 1895197.49 73619.93
Configurations:
(changes since patch 6)
cat mixed.sh
modprobe brd rd_nr=2 rd_size=56623104
swapoff -a
mkswap /dev/ram0
swapon /dev/ram0
mkfs.ext4 /dev/ram1
mount -t ext4 /dev/ram1 /mnt
memtier_benchmark -S /var/run/memcached/memcached.sock \
-P memcache_binary -n allkeys --key-minimum=1 \
--key-maximum=50000000 --key-pattern=P:P -c 1 -t 36 \
--ratio 1:0 --pipeline 8 -d 2000
fio -name=mglru --numjobs=36 --directory=/mnt --size=1408m \
--buffered=1 --ioengine=io_uring --iodepth=128 \
--iodepth_batch_submit=32 --iodepth_batch_complete=32 \
--rw=randread --random_distribution=random --norandommap \
--time_based --ramp_time=10m --runtime=90m --group_reporting &
pid=$!
sleep 200
memtier_benchmark -S /var/run/memcached/memcached.sock \
-P memcache_binary -n allkeys --key-minimum=1 \
--key-maximum=50000000 --key-pattern=R:R -c 1 -t 36 \
--ratio 0:1 --pipeline 8 --randomize --distinct-client-seed
kill -INT $pid
wait
Client benchmark results:
no change (CONFIG_MEMCG=n)
Moto-CRs-Fixed:(CR)
Link: https://lore.kernel.org/r/20220309021230.721028-10-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I0641467dbd7c5ba0645602cec7fe8d6fdb750edb
Reviewed-on: https://gerrit.mot.com/2268701
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
40f2a1fb74 |
FROMLIST: mm: multi-gen LRU: support page table walks
To further exploit spatial locality, the aging prefers to walk page
tables to search for young PTEs and promote hot pages. A kill switch
will be added in the next patch to disable this behavior. When
disabled, the aging relies on the rmap only.
NB: this behavior has nothing similar with the page table scanning in
the 2.4 kernel [1], which searches page tables for old PTEs, adds cold
pages to swapcache and unmaps them.
To avoid confusion, the term "iteration" specifically means the
traversal of an entire mm_struct list; the term "walk" will be applied
to page tables and the rmap, as usual.
An mm_struct list is maintained for each memcg, and an mm_struct
follows its owner task to the new memcg when this task is migrated.
Given an lruvec, the aging iterates lruvec_memcg()->mm_list and calls
walk_page_range() with each mm_struct on this list to promote hot
pages before it increments max_seq.
When multiple page table walkers iterate the same list, each of them
gets a unique mm_struct; therefore they can run concurrently. Page
table walkers ignore any misplaced pages, e.g., if an mm_struct was
migrated, pages it left in the previous memcg will not be promoted
when its current memcg is under reclaim. Similarly, page table walkers
will not promote pages from nodes other than the one under reclaim.
This patch uses the following optimizations when walking page tables:
1. It tracks the usage of mm_struct's between context switches so that
page table walkers can skip processes that have been sleeping since
the last iteration.
2. It uses generational Bloom filters to record populated branches so
that page table walkers can reduce their search space based on the
query results, e.g., to skip page tables containing mostly holes or
misplaced pages.
3. It takes advantage of the accessed bit in non-leaf PMD entries when
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y.
4. It does not zigzag between a PGD table and the same PMD table
spanning multiple VMAs. IOW, it finishes all the VMAs within the
range of the same PMD table before it returns to a PGD table. This
improves the cache performance for workloads that have large
numbers of tiny VMAs [2], especially when CONFIG_PGTABLE_LEVELS=5.
Server benchmark results:
Single workload:
fio (buffered I/O): no change
Single workload:
memcached (anon): +[5.5, 7.5]%
Ops/sec KB/sec
patch1-7: 1014393.57 39455.42
patch1-8: 1078507.59 41949.15
Configurations:
no change
Client benchmark results:
kswapd profiles:
patch1-7
45.54% lzo1x_1_do_compress (real work)
9.56% page_vma_mapped_walk
6.70% _raw_spin_unlock_irq
2.78% ptep_clear_flush
2.47% do_raw_spin_lock
2.22% __zram_bvec_write
1.87% lru_gen_look_around
1.78% memmove
1.77% obj_malloc
1.44% free_unref_page_list
patch1-8
47.02% lzo1x_1_do_compress (real work)
6.73% page_vma_mapped_walk
6.14% _raw_spin_unlock_irq
3.39% walk_pte_range
2.63% ptep_clear_flush
2.29% __zram_bvec_write
2.10% do_raw_spin_lock
1.81% memmove
1.73% obj_malloc
1.53% free_unref_page_list
Configurations:
no change
Moto-CRs-Fixed:(CR)
[1] https://lwn.net/Articles/23732/
[2] https://source.android.com/devices/tech/debug/scudo
Link: https://lore.kernel.org/r/20220309021230.721028-9-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I5a3c97cf8ebf8d65d5f9528cd979a637c190053e
Reviewed-on: https://gerrit.mot.com/2268700
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
6ad1e20b6f |
FROMLIST: mm: multi-gen LRU: exploit locality in rmap
Searching the rmap for PTEs mapping each page on an LRU list (to test
and clear the accessed bit) can be expensive because pages from
different VMAs (PA space) are not cache friendly to the rmap (VA
space). For workloads mostly using mapped pages, the rmap has a high
CPU cost in the reclaim path.
This patch exploits spatial locality to reduce the trips into the
rmap. When shrink_page_list() walks the rmap and finds a young PTE, a
new function lru_gen_look_around() scans at most BITS_PER_LONG-1
adjacent PTEs. On finding another young PTE, it clears the accessed
bit and updates the gen counter of the page mapped by this PTE to
(max_seq%MAX_NR_GENS)+1.
Server benchmark results:
Single workload:
fio (buffered I/O): no change
Single workload:
memcached (anon): +[4, 6]%
Ops/sec KB/sec
patch1-6: 964656.80 37520.88
patch1-7: 1014393.57 39455.42
Configurations:
no change
Client benchmark results:
kswapd profiles:
patch1-6
36.13% lzo1x_1_do_compress (real work)
19.16% page_vma_mapped_walk
6.55% _raw_spin_unlock_irq
4.02% do_raw_spin_lock
2.32% anon_vma_interval_tree_iter_first
2.11% ptep_clear_flush
1.76% __zram_bvec_write
1.64% folio_referenced_one
1.40% memmove
1.35% obj_malloc
patch1-7
45.54% lzo1x_1_do_compress (real work)
9.56% page_vma_mapped_walk
6.70% _raw_spin_unlock_irq
2.78% ptep_clear_flush
2.47% do_raw_spin_lock
2.22% __zram_bvec_write
1.87% lru_gen_look_around
1.78% memmove
1.77% obj_malloc
1.44% free_unref_page_list
Configurations:
no change
Moto-CRs-Fixed:(CR)
Link: https://lore.kernel.org/r/20220309021230.721028-8-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I9a290343840f3cf925c891c8e360c7cdc24ffb9c
Reviewed-on: https://gerrit.mot.com/2268699
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
6a1feddd23 |
FROMLIST: mm: multi-gen LRU: minimal implementation
To avoid confusion, the terms "promotion" and "demotion" will be
applied to the multi-gen LRU, as a new convention; the terms
"activation" and "deactivation" will be applied to the active/inactive
LRU, as usual.
The aging produces young generations. Given an lruvec, it increments
max_seq when max_seq-min_seq+1 approaches MIN_NR_GENS. The aging
promotes hot pages to the youngest generation when it finds them
accessed through page tables; the demotion of cold pages happens
consequently when it increments max_seq. The aging has the complexity
O(nr_hot_pages), since it is only interested in hot pages. Promotion
in the aging path does not require any LRU list operations, only the
updates of the gen counter and lrugen->nr_pages[]; demotion, unless as
the result of the increment of max_seq, requires LRU list operations,
e.g., lru_deactivate_fn().
The eviction consumes old generations. Given an lruvec, it increments
min_seq when the lists indexed by min_seq%MAX_NR_GENS become empty. A
feedback loop modeled after the PID controller monitors refaults over
anon and file types and decides which type to evict when both types
are available from the same generation.
Each generation is divided into multiple tiers. Tiers represent
different ranges of numbers of accesses through file descriptors. A
page accessed N times through file descriptors is in tier
order_base_2(N). Tiers do not have dedicated lrugen->lists[], only
bits in page->flags. In contrast to moving across generations, which
requires the LRU lock, moving across tiers only involves operations on
page->flags. The feedback loop also monitors refaults over all tiers
and decides when to protect pages in which tiers (N>1), using the
first tier (N=0,1) as a baseline. The first tier contains single-use
unmapped clean pages, which are most l(CR) the best choices. The
eviction moves a page to the next generation, i.e., min_seq+1, if the
feedback loop decides so. This approach has the following advantages:
1. It removes the cost of activation in the buffered access path by
inferring whether pages accessed multiple times through file
descriptors are statistically hot and thus worth protecting in the
eviction path.
2. It takes pages accessed through page tables into account and avoids
overprotecting pages accessed multiple times through file
descriptors. (Pages accessed through page tables are in the first
tier, since N=0.)
3. More tiers provide better protection for pages accessed more than
twice through file descriptors, when under heavy buffered I/O
workloads.
Server benchmark results:
Single workload:
fio (buffered I/O): +[38, 40]%
IOPS BW
5.18-ed4643521e6a: 2547k 9989MiB/s
patch1-6: 3540k 13.5GiB/s
Single workload:
memcached (anon): +[103, 107]%
Ops/sec KB/sec
5.18-ed4643521e6a: 469048.66 18243.91
patch1-6: 964656.80 37520.88
Configurations:
CPU: two Xeon 6154
Mem: total 256G
Node 1 was only used as a ram disk to reduce the variance in the
results.
patch drivers/block/brd.c <<EOF
99,100c99,100
< gfp_flags = GFP_NOIO | __GFP_ZERO | __GFP_HIGHMEM;
< page = alloc_page(gfp_flags);
---
> gfp_flags = GFP_NOIO | __GFP_ZERO | __GFP_HIGHMEM | __GFP_THISNODE;
> page = alloc_pages_node(1, gfp_flags, 0);
EOF
cat >>/etc/systemd/system.conf <<EOF
CPUAthene=numa
NUMAPolicy=bind
NUMAMask=0
EOF
cat >>/etc/memcached.conf <<EOF
-m 184320
-s /var/run/memcached/memcached.sock
-a 0766
-t 36
-B binary
EOF
cat fio.sh
modprobe brd rd_nr=1 rd_size=113246208
swapoff -a
mkfs.ext4 /dev/ram0
mount -t ext4 /dev/ram0 /mnt
mkdir /sys/fs/cgroup/user.slice/test
echo 38654705664 >/sys/fs/cgroup/user.slice/test/memory.max
echo $$ >/sys/fs/cgroup/user.slice/test/cgroup.procs
fio -name=mglru --numjobs=72 --directory=/mnt --size=1408m \
--buffered=1 --ioengine=io_uring --iodepth=128 \
--iodepth_batch_submit=32 --iodepth_batch_complete=32 \
--rw=randread --random_distribution=random --norandommap \
--time_based --ramp_time=10m --runtime=5m --group_reporting
cat memcached.sh
modprobe brd rd_nr=1 rd_size=113246208
swapoff -a
mkswap /dev/ram0
swapon /dev/ram0
memtier_benchmark -S /var/run/memcached/memcached.sock \
-P memcache_binary -n allkeys --key-minimum=1 \
--key-maximum=65000000 --key-pattern=P:P -c 1 -t 36 \
--ratio 1:0 --pipeline 8 -d 2000
memtier_benchmark -S /var/run/memcached/memcached.sock \
-P memcache_binary -n allkeys --key-minimum=1 \
--key-maximum=65000000 --key-pattern=R:R -c 1 -t 36 \
--ratio 0:1 --pipeline 8 --randomize --distinct-client-seed
Client benchmark results:
kswapd profiles:
5.18-ed4643521e6a
39.56% page_vma_mapped_walk
19.32% lzo1x_1_do_compress (real work)
7.18% do_raw_spin_lock
4.23% _raw_spin_unlock_irq
2.26% vma_interval_tree_subtree_search
2.12% vma_interval_tree_iter_next
2.11% folio_referenced_one
1.90% anon_vma_interval_tree_iter_first
1.47% ptep_clear_flush
0.97% __anon_vma_interval_tree_subtree_search
patch1-6
36.13% lzo1x_1_do_compress (real work)
19.16% page_vma_mapped_walk
6.55% _raw_spin_unlock_irq
4.02% do_raw_spin_lock
2.32% anon_vma_interval_tree_iter_first
2.11% ptep_clear_flush
1.76% __zram_bvec_write
1.64% folio_referenced_one
1.40% memmove
1.35% obj_malloc
Configurations:
CPU: single Snapdragon 7c
Mem: total 4G
Chrome OS MemoryPressure [1]
[1] https://chromium.googlesource.com/chromiumos/platform/tast-tests/
Moto-CRs-Fixed:(CR)
Link: https://lore.kernel.org/r/20220309021230.721028-7-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I3fe4850006d7984cd9f4fd46134b826609dc2f86
Reviewed-on: https://gerrit.mot.com/2268698
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
81b0532ae5 |
FROMLIST: mm: multi-gen LRU: groundwork
Evictable pages are divided into multiple generations for each lruvec. The youngest generation number is stored in lrugen->max_seq for both anon and file types as they are aged on an equal footing. The oldest generation numbers are stored in lrugen->min_seq[] separately for anon and file types as clean file pages can be evicted regardless of swap constraints. These three variables are monotonically increasing. Generation numbers are truncated into order_base_2(MAX_NR_GENS+1) bits in order to fit into the gen counter in page->flags. Each truncated generation number is an index to lrugen->lists[]. The sliding window technique is used to track at least MIN_NR_GENS and at most MAX_NR_GENS generations. The gen counter stores a value within [1, MAX_NR_GENS] while a page is on one of lrugen->lists[]. Otherwise it stores 0. There are two conceptually independent procedures: "the aging", which produces young generations, and "the eviction", which consumes old generations. They form a closed-loop system, i.e., "the page reclaim". Both procedures can be invoked from userspace for the purposes of working set estimation and proactive reclaim. These features are required to optimize job scheduling (bin packing) in data centers. The variable size of the sliding window is designed for such use cases [1][2]. To avoid confusion, the terms "hot" and "cold" will be applied to the multi-gen LRU, as a new convention; the terms "active" and "inactive" will be applied to the active/inactive LRU, as usual. The protection of hot pages and the selection of cold pages are based on page access channels and patterns. There are two access channels: one through page tables and the other through file descriptors. The protection of the former channel is by design stronger because: 1. The uncertainty in determining the access patterns of the former channel is higher due to the approximation of the accessed bit. 2. The cost of evicting the former channel is higher due to the TLB flushes required and the l(CR) of encountering the dirty bit. 3. The penalty of underprotecting the former channel is higher because applications usually do not prepare themselves for major page faults l(CR) they do for blocked I/O. E.g., GUI applications commonly use dedicated I/O threads to avoid blocking the rendering threads. There are also two access patterns: one with temporal locality and the other without. For the reasons listed above, the former channel is assumed to follow the former pattern unless VM_SEQ_READ or VM_RAND_READ is present; the latter channel is assumed to follow the latter pattern unless outlying refaults have been observed [3][4]. The next patch will address the "outlying refaults". Three macros, i.e., LRU_REFS_WIDTH, LRU_REFS_PGOFF and LRU_REFS_MASK, used later are added in this patch to make the entire patchset less diffy. A page is added to the youngest generation on faulting. The aging needs to check the accessed bit at least twice before handing this page over to the eviction. The first check takes care of the accessed bit set on the initial fault; the second check makes sure this page has not been used since then. This protocol, AKA second chance, requires a minimum of two generations, hence MIN_NR_GENS. [1] https://dl.acm.org/doi/10.1145/3297858.3304053 [2] https://dl.acm.org/doi/10.1145/3503222.3507731 [3] https://lwn.net/Articles/495543/ [4] https://lwn.net/Articles/815342/ Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-6-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: I333ec6a1d2abfa60d93d6adc190ed3eefe441512 Reviewed-on: https://gerrit.mot.com/2268697 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
62f7a9e8c9 |
FROMLIST: mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
Some architectures support the accessed bit in non-leaf PMD entries,
e.g., x86 sets the accessed bit in a non-leaf PMD entry when using it
as part of linear address translation [1]. Page table walkers that
clear the accessed bit may use this capability to reduce their search
space.
Note that:
1. Although an inline function is preferable, this capability is added
as a configuration option for consistency with the existing macros.
2. Due to the little interest in other varieties, this capability was
only tested on Intel and AMD CPUs.
[1]: Intel 64 and IA-32 Architectures Software Developer's Manual
Volume 3 (June 2021), section 4.8
Moto-CRs-Fixed:(CR)
Link: https://lore.kernel.org/r/20220309021230.721028-3-yuzhao@google.com/
Signed-off-by: Yu Zhao <yuzhao@google.com>
Reviewed-by: Barry Song <baohua@kernel.org>
Acked-by: Brian Geffon <bgeffon@google.com>
Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Acked-by: Steven Barrett <steven@liquorix.net>
Acked-by: Suleiman Souhlal <suleiman@google.com>
Tested-by: Daniel Byrne <djbyrne@mtu.edu>
Tested-by: Donald Carr <d@chaos-reins.com>
Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Tested-by: Shuang Zhai <szhai2@cs.rochester.edu>
Tested-by: Sofia Trinh <sofia.trinh@edi.works>
Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
Bug: 228114874
Change-Id: I73f84a21fd315192eaa3e6443334ed1bccb4e99e
Reviewed-on: https://gerrit.mot.com/2268696
SME-Granted: SME Approvals Granted
SLTApproved: Slta Waiver
Tested-by: Jira Key
Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com>
Submit-Approved: Jira Key
|
||
|
Yu Zhao
|
d024618471 |
FROMLIST: mm: x86, arm64: add arch_has_hw_pte_young()
Some architectures automatically set the accessed bit in PTEs, e.g., x86 and arm64 v8.2. On architectures that do not have this capability, clearing the accessed bit in a PTE usually triggers a page fault following the TLB miss of this PTE (to emulate the accessed bit). Being aware of this capability can help make better decisions, e.g., whether to spread the work out over a period of time to reduce bursty page faults when trying to clear the accessed bit in many PTEs. Note that theoretically this capability can be unreliable, e.g., hotplugged CPUs might be different from builtin ones. Therefore it should not be used in architecture-independent code that involves correctness, e.g., to determine whether TLB flushes are required (in combination with the accessed bit). Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/r/20220309021230.721028-2-yuzhao@google.com/ Signed-off-by: Yu Zhao <yuzhao@google.com> Reviewed-by: Barry Song <baohua@kernel.org> Acked-by: Brian Geffon <bgeffon@google.com> Acked-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org> Acked-by: Oleksandr Natalenko <oleksandr@natalenko.name> Acked-by: Steven Barrett <steven@liquorix.net> Acked-by: Suleiman Souhlal <suleiman@google.com> Acked-by: Will Deacon <will@kernel.org> Tested-by: Daniel Byrne <djbyrne@mtu.edu> Tested-by: Donald Carr <d@chaos-reins.com> Tested-by: Holger Hoffstätte <holger@applied-asynchrony.com> Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru> Tested-by: Shuang Zhai <szhai2@cs.rochester.edu> Tested-by: Sofia Trinh <sofia.trinh@edi.works> Tested-by: Vaibhav Jain <vaibhav@linux.ibm.com> Bug: 228114874 Change-Id: Ie81175d7e0d239f688d31487b298cf9b4fb66707 Reviewed-on: https://gerrit.mot.com/2268695 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
d527bf664c |
BACKPORT: include/linux/mm_inline.h: fold page_lru_base_type() into its sole caller
We've removed all other references to this function. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/linux-mm/20201207220949.830352-9-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-9-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> Reviewed-by: Alex Shi <alex.shi@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <guro@fb.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit c1770e34f3e7640887d8129fc05d13fe17101301) Bug: 228114874 Change-Id: If229fa7a09e5be79cc28dc5a780b900e69f4ce64 Reviewed-on: https://gerrit.mot.com/2268694 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
044fa650a0 |
UPSTREAM: mm: VM_BUG_ON lru page flags
Move scattered VM_BUG_ONs to two essential places that cover all lru list additions and deletions. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/linux-mm/20201207220949.830352-8-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-8-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> Cc: Alex Shi <alex.shi@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <guro@fb.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit bc7112719e1e80e4208eef3fc9bd8d2b6c263e7d) Bug: 228114874 Change-Id: I950ad4171f973c740d9fc3778d44efc020d0e12c Reviewed-on: https://gerrit.mot.com/2268693 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
7bb98cd997 |
BACKPORT: mm: add __clear_page_lru_flags() to replace page_off_lru()
Similar to page_off_lru(), the new function does non-atomic clearing of PageLRU() in addition to PageActive() and PageUnevictable(), on a page that has no references left. If PageActive() and PageUnevictable() are both set, refuse to clear either and leave them to bad_page(). This is a behavior change that is meant to help debug. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/linux-mm/20201207220949.830352-7-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-7-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> Cc: Alex Shi <alex.shi@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <guro@fb.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit 875601796267214f286d3581fe74f2805d060fe8) Bug: 228114874 Change-Id: I0290916fa08277c50e228a8d3f39af67d62ff9d0 Reviewed-on: https://gerrit.mot.com/2268692 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
d2b8328404 |
BACKPORT: mm/swap.c: don't pass "enum lru_list" to del_page_from_lru_list()
The parameter is redundant in the sense that it can be potentially extracted from the "struct page" parameter by page_lru(). We need to make sure that existing PageActive() or PageUnevictable() remains until the function returns. A few places don't conform, and simple reordering fixes them. This patch may have left page_off_lru() seemingly odd, and we'll take care of it in the next patch. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/linux-mm/20201207220949.830352-6-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-6-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> Cc: Alex Shi <alex.shi@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <guro@fb.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit 46ae6b2cc2a47904a368d238425531ea91f3a2a5) Bug: 228114874 Change-Id: I1e14dcbf4111b39cf155ed3512423448865eb324 Reviewed-on: https://gerrit.mot.com/2268691 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |
||
|
Yu Zhao
|
73deefe8e0 |
UPSTREAM: mm/swap.c: don't pass "enum lru_list" to trace_mm_lru_insertion()
The parameter is redundant in the sense that it can be extracted from the "struct page" parameter by page_lru() correctly. Moto-CRs-Fixed:(CR) Link: https://lore.kernel.org/linux-mm/20201207220949.830352-5-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-5-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> Reviewed-by: Alex Shi <alex.shi@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <guro@fb.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit 861404536a3af3c39f1b10959a40def3d8efa2dd) Bug: 228114874 Change-Id: Ia02c0c65dd427a98ffa39e9dc3e2ae701e85fad8 Reviewed-on: https://gerrit.mot.com/2268690 SME-Granted: SME Approvals Granted SLTApproved: Slta Waiver Tested-by: Jira Key Reviewed-by: Xiangpo Zhao <zhaoxp3@motorola.com> Submit-Approved: Jira Key |