• This provides a new implementation, which uses co_positions() to lookup the ast node.
• It has a simpler implementation and better performance.
• Some limitations in the unit tests are removed for 3.11.
  • support for and and or
  • no ambiguities for generators
  • fixes #30

After experimenting a little bit, I found that for python packages, including a requirements.txt in the repo would enable GH to index who's using the package.

The list can be seen on both the main page of the repo or Insights -> Dependency graph -> Dependents

I was able to execute all tests with pytest. However assertion rewriting is still disabled for test_main.py.

But this has already several benefits:

• tox -e py310 -- --sw works
• no need to maintain two test environments

I also converted the test_sample_files to an parametrized test which makes it more useful for pytest --sw

I also don't know how I should handle the timeouts. I removed them for test_sample_files because they are not really useful for single file checks. I kept them for test_module_files because this takes already a bit longer. What's your opinion there?

I wanted to get your feedback before I continue.

generate small samples to improve test coverage.

• The samples where generated with tox -e mutmut (which calls tests/mutmut_workflow.py, which class tests/generate_small_sample.py).
• The samples are placed in tests/small_samples

Every sample is the result of a code mutation detected by mutmut which was not covered by any other test (excluding the long running tests).

generate_small_sample.py tries to find a source file with some code which fails with the mutated implementation and removes code from this file with pysource_minimize.

I have also found some issues:

• The ExceptHandler cleanup is now an KnownIssue. Before it was verified as an valid mapping, which is not ok.
• DELETE/STORE_DEREF had some issues.

I had to reimplement the Deadcode analysis. The problem was that I always tried to reimplement the deadcode analysis of the python interpreter, which did not scale very well. The new implementation replaces the node with an sentinel compiles the source and checks if the sentinel is in the bytecode. The downside is that I had to remove the inverse deadcode check, because the new implementation is a bit more complex ( compiles the whole file for every node to test).

Let me know what you think.

closes #55

try to fix #49

I also reported frame.f_lasti and the problem is that the instruction is a CACHE instruction in the deco case (f_lasti == 130).

The Tester case gets the expected instruction and maps the bytecode correctly.

You can find my source here https://github.com/15r10nk/executing/tree/decorator-detection-bug

I could ignore CACHE instructions until now. I will try to figure out what they are good for.

With today's release of Python 3.11.0a3, I suddenly have many failing tests for friendly-traceback that were still passing with 3.11.0a2. It looks like the nodes are no longer properly identified.

For example, if I have

a = (1, 2)(3, 4)

the node was previously identified as (1, 2)(3, 4) whereas now it is the entire line that is identified as the problematic node.

I can provide more examples if needed, but I suspect that your own unit tests would likely give you some results. It might be that the problem is upstream (with asttoken) instead of in executing.

Apologies for not providing a self-contained test with executing for the following, as I don't know how. The following raises a ValueError exception:

class F:
    __slots__ = ["a", "b"]
    a = 1

With Python 3.10 (and prior), the text of the node identified causing the exception is:

class F:

With Python 3.11, we get this:

class F: __slots__ = ["a", "b"] a = 1

This is not a typo: the text of the node identified a single line, with no \n, which is clearly a SyntaxError.

This is the only bug I found when running the unit tests for friendly_traceback with executing 1.1.0 and Python 3.11. Everything still works for me with Python 3.6 to 3.10, although I found that the tests I run take about 2 to 3 times as long as compared with using executing 0.8.3. The slowdown seemed to be already present with executing 1.0.0.

Using friendly, I found that excuting could not locate the correct location of a node causing a problem inside Jupyter notebooks (or Jupyter lab) but could do so with IPython. First, here's a screen capture showing the correct result using IPython, with some additional outputs from print statements inserted for this specific report.

Next, the same code run within a Jupyter notebook

Perhaps this is is caused by the new way that Jupyter runs code, using temporary files instead of using exec() on code.

With the "old" friendly-traceback (before it made use of stack_data), when Jupyter was not using temporary files, I see from screen captures on https://aroberge.github.io/friendly-traceback-docs/docs/html/jupyter.html that the location was correctly picked up.

Would it be possible to upload the sdist in addition to the wheel file on pypi? Currently the conda-forge distribution does not automatically pick up updates unless the sources are present.

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-executing-0.8.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-executing-0.8.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/executing-0.8.3
collected 2 items / 1 error

================================================================================== ERRORS ==================================================================================
___________________________________________________________________ ERROR collecting tests/test_main.py ____________________________________________________________________
executing/executing.py:317: in executing
    args = executing_cache[key]
E   KeyError: (<code object <module> at 0x7f98855a0ea0, file "/home/tkloczko/rpmbuild/BUILD/executing-0.8.3/tests/test_main.py", line 2>, 140293049028256, 418)

During handling of the above exception, another exception occurred:
executing/executing.py:346: in find
    node_finder = NodeFinder(frame, stmts, tree, lasti)
executing/executing.py:636: in __init__
    matching = list(self.matching_nodes(exprs))
executing/executing.py:702: in matching_nodes
    original_index = only(
executing/executing.py:164: in only
    raise NotOneValueFound('Expected one value, found 0')
E   executing.executing.NotOneValueFound: Expected one value, found 0

During handling of the above exception, another exception occurred:
tests/test_main.py:682: in <module>
    assert tester([1, 2, 3]) == [1, 2, 3]
tests/utils.py:40: in __call__
    ex = self.get_executing(inspect.currentframe().f_back)
tests/utils.py:28: in get_executing
    return Source.executing(frame)
executing/executing.py:369: in executing
    args = find(source=cls.for_frame(frame), retry_cache=True)
executing/executing.py:355: in find
    return find(
executing/executing.py:346: in find
    node_finder = NodeFinder(frame, stmts, tree, lasti)
executing/executing.py:636: in __init__
    matching = list(self.matching_nodes(exprs))
executing/executing.py:702: in matching_nodes
    original_index = only(
executing/executing.py:164: in only
    raise NotOneValueFound('Expected one value, found 0')
E   executing.executing.NotOneValueFound: Expected one value, found 0
========================================================================= short test summary info ==========================================================================
ERROR tests/test_main.py - executing.executing.NotOneValueFound: Expected one value, found 0
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 error in 0.44s =============================================================================

For example

def do_func(self):
    frame = sys._getframe()

class A:
    def do_method(self):
        frame = sys._getframe()

Is it possible to know whether frame is inside a regular function call or method call? From https://stackoverflow.com/q/2203424/2142577 it seems this is no way except relying on the convention that self is the first argument of method signature. Do we have a better way?

I also tried using gc.get_referrers(frame.f_code)), but it turns out the code is referrenced by the unbound function, no matter whether it is actually a method.

This is still work in progress.

The goal is an NodeFinder with similar capabilities like PositionNodeFinder.

todo:

• [ ] workaround for https://github.com/python/cpython/issues/100537
• [ ] integrate node verification ( create Verifier which can be used for python 3.11 and 3.10)
• expand to:
  • [ ] 3.9
  • [ ] 3.8
  • [ ] 3.7
  • [ ] 3.6 (if possible)

Obligatory "I'm a huge fan of your work".

Background: I maintain a library called ipyflow, which uses another library I maintain called pyccolo. I noticed that on ipython >= 8.0, which uses executing for better stack traces, ipyflow would have really bad performance regressions the first time after a cell throws some exception.

Eventually I traced it to pyccolo's use of copy.deepcopy in a few places -- performance was bad because the parent pointers that executing added to AST nodes were causing deepcopy to do a lot of extra unnecessary work.

I ended up working around it on the pyccolo side, but I figured you may be interested in this for other libraries that may want to use executing and get surprised when deepcopy has bad perf. The way we maintain deepcopy-ability in pyccolo is to maintain a mapping from id(node) to parent for parent pointers, which avoids setting an attribute on the AST node directly.

Thanks for this great library!

Currently GHA has coverage run --include=executing/executing.py <run tests> which doesn't measure coverage in the recently added files in the executing folder. The original motivation was lines in __init__.py that would never be covered, but that should be handled by a # pragma: no cover comment or something.