forked from mirrors/gecko-dev
6d154c1ed3
Vendoring wheels has three benefits: * There's far less files, so Firefox checkouts will be smaller. * It works around `zipp` not allowing `pip install` from extracted source `tar.gz` files. Now, we should be able to use the pip resolver against vendored packages, which will be needed for future mach virtualenv work. * `./mach vendor python` takes far less time to execute. Since we need the raw Python to be available to add to the `sys.path`, we extract the wheels before putting them in tree. Due to the structure of some wheels being less nested than of a source `tar.gz`, `common_virtualenv_packages` needed to be adjusted accordingly. `install_pip_package()` had to be tweaked as well since you can't `pip install` an extracted wheel. So, we "re-bundle" the wheel before installing from a vendored package. Replace python packages with wheels where possible This contains the vendoring changes caused by the last patch. For reviewing, there's a couple things to note: * A bunch of files are deleted, since there's generally less files in a wheel than in a source archive. * There's a new `.dist-info` directory for each extracted wheel, so expect roughly 5 or 6 new files for each wheel'd package. * There should be no source code changes other than moves from package names changing from having `-` to having `_`. Differential Revision: https://phabricator.services.mozilla.com/D116512
81 lines
3.2 KiB
Python
81 lines
3.2 KiB
Python
# -*- coding: utf-8 -*-
|
|
"""API for traversing the AST nodes. Implemented by the compiler and
|
|
meta introspection.
|
|
"""
|
|
from .nodes import Node
|
|
|
|
|
|
class NodeVisitor(object):
|
|
"""Walks the abstract syntax tree and call visitor functions for every
|
|
node found. The visitor functions may return values which will be
|
|
forwarded by the `visit` method.
|
|
|
|
Per default the visitor functions for the nodes are ``'visit_'`` +
|
|
class name of the node. So a `TryFinally` node visit function would
|
|
be `visit_TryFinally`. This behavior can be changed by overriding
|
|
the `get_visitor` function. If no visitor function exists for a node
|
|
(return value `None`) the `generic_visit` visitor is used instead.
|
|
"""
|
|
|
|
def get_visitor(self, node):
|
|
"""Return the visitor function for this node or `None` if no visitor
|
|
exists for this node. In that case the generic visit function is
|
|
used instead.
|
|
"""
|
|
method = "visit_" + node.__class__.__name__
|
|
return getattr(self, method, None)
|
|
|
|
def visit(self, node, *args, **kwargs):
|
|
"""Visit a node."""
|
|
f = self.get_visitor(node)
|
|
if f is not None:
|
|
return f(node, *args, **kwargs)
|
|
return self.generic_visit(node, *args, **kwargs)
|
|
|
|
def generic_visit(self, node, *args, **kwargs):
|
|
"""Called if no explicit visitor function exists for a node."""
|
|
for node in node.iter_child_nodes():
|
|
self.visit(node, *args, **kwargs)
|
|
|
|
|
|
class NodeTransformer(NodeVisitor):
|
|
"""Walks the abstract syntax tree and allows modifications of nodes.
|
|
|
|
The `NodeTransformer` will walk the AST and use the return value of the
|
|
visitor functions to replace or remove the old node. If the return
|
|
value of the visitor function is `None` the node will be removed
|
|
from the previous location otherwise it's replaced with the return
|
|
value. The return value may be the original node in which case no
|
|
replacement takes place.
|
|
"""
|
|
|
|
def generic_visit(self, node, *args, **kwargs):
|
|
for field, old_value in node.iter_fields():
|
|
if isinstance(old_value, list):
|
|
new_values = []
|
|
for value in old_value:
|
|
if isinstance(value, Node):
|
|
value = self.visit(value, *args, **kwargs)
|
|
if value is None:
|
|
continue
|
|
elif not isinstance(value, Node):
|
|
new_values.extend(value)
|
|
continue
|
|
new_values.append(value)
|
|
old_value[:] = new_values
|
|
elif isinstance(old_value, Node):
|
|
new_node = self.visit(old_value, *args, **kwargs)
|
|
if new_node is None:
|
|
delattr(node, field)
|
|
else:
|
|
setattr(node, field, new_node)
|
|
return node
|
|
|
|
def visit_list(self, node, *args, **kwargs):
|
|
"""As transformers may return lists in some places this method
|
|
can be used to enforce a list as return value.
|
|
"""
|
|
rv = self.visit(node, *args, **kwargs)
|
|
if not isinstance(rv, list):
|
|
rv = [rv]
|
|
return rv
|