Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (14)
Similar to AST: threats and opportunities
Similar to AST: threats and opportunities (20)
Recently uploaded
Recently uploaded (20)
AST: threats and opportunities
- 1. AST Opportunities and Threats ITGM, Dec 2016
- 2. RDD TDD Slap this sh*t & release ITGM, Dec 2016
- 3. Code Test Test Test ITGM, Dec 2016
- 4. Code Test Test Test def f(a,b): return a+b 5 == f(2,3) 0 == f(2,-2) 1 == f(1,0) ITGM, Dec 2016
- 5. Code Test Test Test def f(a,b): return a+b 5 == f(2,3) 0 == f(2,-2) 1 == f(1,0) Test 1e15 == f(1,1e15-1) ITGM, Dec 2016
- 7. >>> def foo(): ... a = 2 ... b = 3 ... return a + b ... ITGM, Dec 2016
- 8. >>> import dis >>> dis.dis(foo) 2 0 LOAD_CONST 1 (2) 3 STORE_FAST 0 (a) 3 6 LOAD_CONST 2 (3) 9 STORE_FAST 1 (b) 4 12 LOAD_FAST 0 (a) 15 LOAD_FAST 1 (b) 18 BINARY_ADD 19 RETURN_VALUE ITGM, Dec 2016
- 11. ITGM, Dec 2016
- 16. Lib/ast.py import ast ITGM, Dec 2016
- 17. • AST • Add, And, Assert … • NodeVisitor • NodeTransformer ITGM, Dec 2016
- 18. class DepthVisitor(ast.NodeVisitor): depth = 0 def generic_visit(self, node): parent = node.parent if parent: while parent: self.depth += 1 parent = parent.parent ast.NodeVisitor.generic_visit(self, node) ITGM, Dec 2016
- 19. • parse • copy_location • fix_missing_locations • dump • increment_lineno ITGM, Dec 2016
- 20. Модификация AST ITGM, Dec 2016
- 21. class HackedImporter: def load_module(self, name): # do magic stuff return module sys.path_hook.insert(0, HackedImporter) ITGM, Dec 2016
- 23. class ReturnIncrement(ast.NodeTransformer): def visit_Return(self, node): node.value = (ast.BinOp( left=node.value, op=ast.Add(), right=ast.Num(n=1) ) return node ITGM, Dec 2016
- 24. def transform_with(transformer): def transform_decorator(func): node = func_to_node(func) node = transformer().visit(node) node = ast.fix_missing_locations(node) func = node_to_func(node, func) return func return transform_decorator ITGM, Dec 2016
- 25. @transform_with(ReturnIncrementer) def f(a,b): return a+b f(2,2) # 5 ITGM, Dec 2016
- 26. Threats • Debug glitch • Errors with multiple applies • Strange errors • Compiler version dependence ITGM, Dec 2016
- 27. >>> parseprint("func(a, b=c, *d, **e)") # Python 3.4 Module(body=[ Expr(value=Call(func=Name(id='func', ctx=Load()), args=[Name(id='a', ctx=Load())], keywords=[keyword(arg='b', value=Name(id='c', ctx=Load()))], starargs=Name(id='d', ctx=Load()), # gone in 3.5 kwargs=Name(id='e', ctx=Load()))), # gone in 3.5 ]) >>> parseprint("func(a, b=c, *d, **e)") # Python 3.5 Module(body=[ Expr(value=Call(func=Name(id='func', ctx=Load()), args=[ Name(id='a', ctx=Load()), Starred(value=Name(id='d', ctx=Load()), ctx=Load()) # new in 3.5 ], keywords=[ keyword(arg='b', value=Name(id='c', ctx=Load())), keyword(arg=None, value=Name(id='e', ctx=Load())) # new in 3.5 ])) ]) ITGM, Dec 2016
- 28. Opportunities • Constant propagation • Autologger • Autoprofiler • Macros (karnickel) • Extend language • MacroPy • PonyORM • Linters (auto codereview) ITGM, Dec 2016
- 29. >>> from macropy.tracing import macros, trace >>> trace[[x*2 for x in range(3)]] range(3) -> [0, 1, 2] x*2 -> 0 x*2 -> 2 x*2 -> 4 x*2 for x in range(3) -> [0, 2, 4] [0, 2, 4] Macros ITGM, Dec 2016
- 30. welcome = gettext("Welcome, {}!").format(user_name) welcome = gettext("Welcome, {}!".format(user_name)) Linters ITGM, Dec 2016
- 31. https://github.com/edx/edx-lint/blob/master/edx_lint/pylint/ i18n_check.py Linters ITGM, Dec 2016
- 32. TRANSLATION_FUNCTIONS = set([ '_', 'gettext', 'ngettext', 'ngettext_lazy', 'npgettext', 'npgettext_lazy', 'pgettext', 'pgettext_lazy', 'ugettext', 'ugettext_lazy', 'ugettext_noop', 'ungettext', 'ungettext_lazy', ]) def visit_callfunc(self, node): if not isinstance(node.func, astroid.Name): # It isn't a simple name, can't deduce what function it is. return if node.func.name not in self.TRANSLATION_FUNCTIONS: # Not a function we care about. return if not self.linter.is_message_enabled(self.MESSAGE_ID): return first = node.args[0] if isinstance(first, astroid.Const): if isinstance(first.value, basestring): # The first argument is a constant string! All is well! return # Bad! self.add_message(self.MESSAGE_ID, args=node.func.name, node=node) ITGM, Dec 2016
- 33. Tools • astdump • astviewer • ast_tool_box • astroid • baron • redbaron • astor • meta • astmonkey ITGM, Dec 2016
- 34. Recommendation unit tests system ITGM, Dec 2016
- 38. Code1 Test1 Code1 Test2 Code1 Test3 Code2 Test1 Code2 Test2 ITGM, Dec 2016
- 39. def test_default_ordering(self): class OrderingListView(generics.ListAPIView): queryset = OrderingFilterModel.objects.all() serializer_class = OrderingFilterSerializer filter_backends = (filters.OrderingFilter,) ordering = ('title',) ordering_fields = ('text',) view = OrderingListView.as_view() request = factory.get('') response = view(request) self.assertEqual( response.data, [ {'id': 3, 'title': 'xwv', 'text': 'cde'}, {'id': 2, 'title': 'yxw', 'text': 'bcd'}, {'id': 1, 'title': 'zyx', 'text': 'abc'}, ] ) # not in context ITGM, Dec 2016
- 40. def fact(a): if a == 0: return 1 return a * fact(a - 1) def test_fact(self): self.assertTrue(fact(0) == 1) self.assertTrue(fact(1) == 1) self.assertTrue(fact(2) == 2) self.assertTrue(fact(3) == 6) self.assertTrue(fact(4) == 24) ITGM, Dec 2016
- 43. ITGM, Dec 2016
- 44. ITGM, Dec 2016
- 45. ITGM, Dec 2016
- 46. ITGM, Dec 2016
- 47. ITGM, Dec 2016
- 48. ITGM, Dec 2016
- 51. Child Edge Parent Freq Num left BinOp 12 Num value Assign 20 ITGM, Dec 2016
- 53. Builtin function ITGM, Dec 2016
- 58. Improve search modules names as features ITGM, Dec 2016
- 59. Import github projects unit-test monitoring ITGM, Dec 2016
- 61. Performance optimization map -> list comprehension *pylint plugin ITGM, Dec 2016
- 62. Other languages JavaScript ITGM, Dec 2016
- 63. Real TDD tests -> code generation ITGM, Dec 2016