gh-141770: Annotate anonymous mmap usage if "-X dev" is used #142079

@SpecLad Just out of curiosity, passing the build flag through ./configure --enable-annotate-mmap would be fine to you?
I would like to follow Ruby's approach but disable this flag by default unless many disto actually want this flag.

SpecLad · 2025-11-29T15:55:34Z

To be clear, by "option" I really mean a runtime option. 🙂 It would be pretty annoying to have to rebuild Python just for this. If you compare with other implementations, none of them require a compile-time option:

C (glibc) - off by default, can be enabled with environment variable;
Go - on by default, can be disabled with environment variable;
Ruby - on permanently (unless I missed something).

TBH, I don't see why Python shouldn't also permanently enable this, but I would settle for "off by default, enabled with environment variable or -X option".

picnixz · 2025-11-29T15:57:55Z

It's a bit annoying that this would only be supported on Linux. Do we have an option that isn't supported everywhere?

zooba · 2025-12-03T19:29:07Z

Include/internal/pycore_mmap.h

+#  include <sys/prctl.h>
+#endif
+
+static inline int


Surely this isn't worth putting into a macro? As a regular function, it will be fully internal, and PGO can inline the parts of it that are appropriate.

I prefer static inline functions. I don't see the advantage of using a macro. Inlining or not shouldn't make any difference on performance.

corona10 · 2025-12-04T12:22:09Z

@zooba I've updated PR. PTAL :)

vstinner · 2025-12-04T13:47:48Z

Include/internal/pycore_mmap.h

+#if defined(HAVE_PR_SET_VMA_ANON_NAME) && defined(__linux__)
+#  define _PyAnnotateMemoryMap(addr, size, name) \
+    do { \
+        if (_Py_GetConfig()->dev_mode) { \


I understood that it should always be enabled if Python is built in debug mode (if Py_DEBUG macro is defined).

I prefer consistent behavior

I considered the following options:

Always enable it for both debug and release builds (as long as the build flag is enabled).

Enable it only when -X dev is passed. (current choice)

To me, any other combination feels weird (e.g enabling it unconditionally in debug builds but requiring -X dev for release build)

There is another code paths which is enabled if the Development Mode is enabled or Python is built in debug mode: checking errors argument in str.encode(encoding, errors), PyUnicode_Decode() and PyUnicode_FromEncodedObject().

In the past, a similar thing was done for errors on closing a file (io_finalize() in Modules/_io/iobase.c). Since Python 3.13, close errors on I/O are now always logged as unraisable exceptions.

Accepted the suggestion :)

picnixz · 2025-12-05T10:30:21Z

Include/internal/pycore_mmap.h

+#endif
+
+#if defined(HAVE_PR_SET_VMA_ANON_NAME) && defined(__linux__) && defined(Py_DEBUG)
+#  define _PyAnnotateMemoryMap(addr, size, name) \


Do we really need macros? it is just to avoid the (unsigned long) cast? If so, I would suggest having a static inline function (but you can have a macro for the no-op case to avoid an empty function) and leave the rest to the compiler. Or if you want to keep a macro, please align the \ (see PEP-7).

I would also prefer a static inline function for cleaner code.

vstinner · 2025-12-05T11:48:58Z

Include/internal/pycore_mmap.h

+#endif
+
+#if defined(HAVE_PR_SET_VMA_ANON_NAME) && defined(__linux__) && defined(Py_DEBUG)
+#  define _PyAnnotateMemoryMap(addr, size, name) \


I would also prefer a static inline function for cleaner code.

Doc/whatsnew/3.15.rst

Modules/mmapmodule.c

Objects/obmalloc.c

Co-authored-by: Victor Stinner <[email protected]>

corona10

@vstinner @picnixz I 've applied all your suggestion.

picnixz · 2025-12-05T15:20:02Z

Doc/whatsnew/3.15.rst

  modules that are missing or packaged separately.
  (Contributed by Stan Ulbrych and Petr Viktorin in :gh:`139707`.)

+* Annotating anonymous mmap usage is now supported if Linux kernel supports ``PR_SET_VMA_ANON_NAME`` (Linux 5.17 or newer).


* Annotating anonymous mmap usage is now supported if Linux kernel supports :manpage:`PR_SET_VMA_ANON_NAME <PR_SET_VMA(2const)>` (Linux 5.17 or newer).

I don't know under which manpage entry it is, but if you locally run Sphinx, you should be able to find the correct one. That way, users would have a link to click on. If there isn't a manpage, nevermind.

Misc/NEWS.d/next/Core_and_Builtins/2025-11-29-18-14-28.gh-issue-141770.JURnvg.rst

picnixz · 2025-12-05T15:22:17Z

Include/internal/pycore_mmap.h

+    }
+#endif
+    assert(strlen(name) < 80);
+    prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, (unsigned long)(addr), size, name);


Do you want to restore the previous errno maybe? (I actually don't know whether we're consistent in "ignoring" errnos).

Accepted :)

…e-141770.JURnvg.rst Co-authored-by: Bénédikt Tran <[email protected]>

Misc/NEWS.d/next/Core_and_Builtins/2025-11-29-18-14-28.gh-issue-141770.JURnvg.rst

…e-141770.JURnvg.rst Co-authored-by: Victor Stinner <[email protected]>

corona10 and others added 3 commits November 29, 2025 01:14

pythongh-141770: Annotate anonymous mmap only for debug build

0e3933d

Apply annotation feature to modules

8b7023e

Add NEWS.d

816d962

corona10 requested review from AA-Turner, brandtbucher, diegorusso, emmatyping, ericsnowcurrently, erlend-aasland and savannahostrowski as code owners November 29, 2025 09:16

bedevere-app bot added the awaiting core review label Nov 29, 2025

bedevere-app bot mentioned this pull request Nov 29, 2025

Name arenas allocated by mmap in _PyMem_ArenaAlloc for debugging #141770

Open

picnixz reviewed Nov 29, 2025

View reviewed changes

Include/internal/pycore_pymem.h Outdated Show resolved Hide resolved

Include/internal/pycore_pymem.h Outdated Show resolved Hide resolved

corona10 and others added 3 commits November 29, 2025 09:30

nit

979ce02

Update Include/internal/pycore_pymem.h

b3d8547

Co-authored-by: Bénédikt Tran <[email protected]>

Address code review

0c5a495

corona10 requested a review from picnixz November 29, 2025 10:13

picnixz reviewed Nov 29, 2025

View reviewed changes

Include/internal/pycore_pymem.h Outdated Show resolved Hide resolved

Address code review

e757dd9

picnixz approved these changes Nov 29, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Nov 29, 2025

corona10 added 2 commits December 4, 2025 03:27

fix

c0e08b6

Update PC

e66e650

corona10 requested a review from a team as a code owner December 3, 2025 18:29

corona10 changed the title ~~gh-141770: Annotate anonymous mmap usage in debug builds only~~ gh-141770: Annotate anonymous mmap usage if "-X dev" is used Dec 3, 2025

corona10 added 2 commits December 4, 2025 03:31

Update

b0ca751

Update Whats News

c9138de

zooba reviewed Dec 3, 2025

View reviewed changes

corona10 added 2 commits December 4, 2025 21:03

Merge remote-tracking branch 'upstream/main' into pythongh-141770

8f77e0d

Address code review

8706fce

corona10 requested a review from zooba December 4, 2025 12:08

corona10 added 2 commits December 4, 2025 21:11

lint

8ce8281

Update

d71a744

vstinner reviewed Dec 4, 2025

View reviewed changes

Address code review

4dd1ee4

corona10 requested a review from vstinner December 4, 2025 18:43

picnixz reviewed Dec 5, 2025

View reviewed changes

vstinner reviewed Dec 5, 2025

View reviewed changes

corona10 and others added 3 commits December 5, 2025 23:03

Apply suggestions from code review

fee757e

Co-authored-by: Victor Stinner <[email protected]>

Update Objects/obmalloc.c

a240b3c

Co-authored-by: Victor Stinner <[email protected]>

Address code review

805c539

corona10 requested a review from vstinner December 5, 2025 14:08

corona10 commented Dec 5, 2025

View reviewed changes

picnixz approved these changes Dec 5, 2025

View reviewed changes

corona10 and others added 4 commits December 6, 2025 00:42

Update Misc/NEWS.d/next/Core_and_Builtins/2025-11-29-18-14-28.gh-issu…

795d31c

…e-141770.JURnvg.rst Co-authored-by: Bénédikt Tran <[email protected]>

Address code review

be4d550

Address code review

290864a

nit

b287e0c

vstinner reviewed Dec 6, 2025

View reviewed changes

Misc/NEWS.d/next/Core_and_Builtins/2025-11-29-18-14-28.gh-issue-141770.JURnvg.rst Outdated Show resolved Hide resolved

Update Misc/NEWS.d/next/Core_and_Builtins/2025-11-29-18-14-28.gh-issu…

1619f95

…e-141770.JURnvg.rst Co-authored-by: Victor Stinner <[email protected]>

Uh oh!

gh-141770: Annotate anonymous mmap usage if "-X dev" is used #142079

Are you sure you want to change the base?

gh-141770: Annotate anonymous mmap usage if "-X dev" is used #142079

Uh oh!

Conversation

corona10 commented Nov 29, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Annotation

Without annotation

Uh oh!

corona10 commented Nov 29, 2025

Uh oh!

corona10 commented Nov 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SpecLad commented Nov 29, 2025

Uh oh!

corona10 commented Nov 29, 2025

Uh oh!

corona10 commented Nov 29, 2025

Uh oh!

SpecLad commented Nov 29, 2025

Uh oh!

picnixz commented Nov 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corona10 commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corona10 Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

corona10 commented Nov 29, 2025 •

edited by bedevere-app bot

Loading

corona10 commented Nov 29, 2025 •

edited

Loading

corona10 commented Dec 4, 2025 •

edited

Loading

corona10 Dec 4, 2025 •

edited

Loading