I have an issue which I believe is caused by hitting the recursion limit inside of a getattr
call. However, I have no idea how to debug this. Take this simple contrived example:
def f(i):
try:
f(i+1)
except RecursionError:
print(f"Reached max recursion depth before breakpoint {i}")
breakpoint()
print(f"Reached max recursion depth after breakpoint {i}")
- The first print happens at
i=996
(or a partial print ati=997
, somewhere around here) - Then there are many more prints as the
breakpoint
function itself fails to work correctly and raises anRecursionError
- at
i=949
we actually get a debugger finally, but this is ofcourse far away from where I can actually play around at the recursion limit
Even just raw print debugging might not work and is probably at least one layer removed from where the problem happens.
Note on the original issue: getattr(self, generated_name)
returns None
despite the object having the name for sure and this having succeeded many times before. Also note: Increasing the recursion limit is not a solution. There are potentially infinite recursions which I am trying to catch with an RecursionError
.
Hmm. My first thought is sys.setrecursionlimit()
but that is itself a call. It might help though?
Well, even if it works, then I am no longer at recursion limit and can no longer observe the behavior I want to observe…
Oh, but I also randomly figured out the original issue… This is quite the edge case xd
name
isn’t astr
instances, but a subclass ofstr
(calledToken
) with a custom__eq__
, but one that does defer tostr.__eq__
if the other isn’t an instance ofToken
getattr(self, name)
potentially falls through to a__getattr__
, which provides a default value.
So I guess what happens is:
getattr(self, name)
is called, which tries to look upname
inself.__dict__
. This calls__eq__
, which causes anRecursionError
because of the call tostr.__eq__
(or earlier, doesn’t really matter)- For some reason,
getattr
doesn’t propagate this error, treats it as anAttributeError
and callsself.__getattr__
, which succeeds, returning the wrong value
This should definitely be fixed, I will try to make a proper reproducer and report it to CPython. (In my case a workaround of not using the subclass is ok)
But the OP question is IMO still interesting, if someone has a nice idea, I am interested.
Sorry, lemme clarify. I wasn’t clear in what I was saying. I meant that using setrecursionlimit inside the exception handler might solve it. Something like this:
def f(i):
try:
f(i+1)
except RecursionError:
sys.setrecursionlimit(sys.getrecursionlimit() + 10)
print(f"Reached max recursion depth before breakpoint {i}")
breakpoint()
rather than as a global change.
The exception handler then gets called 5 times till we are far enough away from the recursion limit so that breakpoint()
works now. But that still means I can’t easily observe the behavior of the recursion limit itself. Ideally, I would like to be able to type print("XYZ")
and instead of it printing XYZ
it would raise an recursion error (which is the caught and displayed like 1/0
would be in a normal shell). But stdlib pdm
is for sure not designed for that, and I am struggling even imaging a python shell that does. I guess dynamically changing the recursion limit for each exec
is a decent approach?