This document provides an overview of how to add new syntax to the Python programming language by modifying the CPython implementation. It begins with terminology related to compilers like grammars, scanners, parsers, abstract syntax trees, and bytecode. It then discusses the general architecture of a bytecode compiler and how CPython's compiler works. The document uses adding a new "unless" statement as a case study, outlining the four main steps: 1) describing the syntax, 2) adding new AST nodes, 3) transforming parses trees to ASTs, and 4) emitting bytecode. It emphasizes examining existing constructs like "if" for examples and provides context on working with the CPython source code.
The document discusses translating .NET bytecode to run on the Parrot virtual machine (VM). It describes:
1) Choosing bytecode translation over other options like modifying compilers or embedding VMs, as it provides better performance and independence from programming languages.
2) The challenges of translating between the stack-based .NET bytecode and register-based Parrot bytecode.
3) The architecture developed to make the translation pluggable and declarative, including a metadata translator, instruction translator, and stack to register mapping modules.
Parrot -- "one bytecode to rule them all"Nuno Carvalho
Parrot is a virtual machine designed to compile and execute bytecode for dynamic languages. It is register-based and object-oriented. The presentation discusses Parrot's design, features like exception handling and garbage collection, and how to build compilers for Parrot using the Parrot Compiler Toolkit.
Strategies to improve embedded Linux application performance beyond ordinary ...André Oriani
he common recipe for performance improvement is to profile an application, identify the most time-consuming routines, and finally select them for optimization. Sometimes that is not enough. Developers may have to look inside the OS searching for performance improvement opportunities. Or they might need to optimize code inside a third party library they do not have access to. For those cases, other strategies shall be used. This presentation reports the experiences of Motorola's Brazilian developers reducing the startup time of an application on Motorola's MOTOMAGX embedded Linux platform. Most of the optimization was performed in the binary loading stage, prior to the execution of the entry point function. This endeavor required use of Linux ABI and Linux Loader going beyond typical bottleneck searching. The presentation will cover prelink, dynamic library loading, tuning of shared objects, and enhancing user experience. A live demo will show the use of prelink and other tools to improve performance of general Linux platforms when libraries are used.
The document provides an overview of Erlang and its features for building scalable and fault-tolerant systems. It discusses how Erlang uses lightweight processes, message passing, and supervision to allow for high concurrency. It also covers how Erlang enables hot code upgrading and distribution across nodes.
Gábor Hojtsy gave a presentation on the history and process of localization and translation for Drupal projects. He discussed how translation strings are extracted from code and stored in .po files to be shared among translators. Localize.drupal.org was created to streamline the translation process by automating tasks like parsing releases and providing a centralized web interface. The site now hosts over 5,600 translated releases and 149,000 translation strings across many projects.
This document provides an overview of Puppet, a system configuration management tool. Puppet uses a declarative language to define system resources and their desired state. It has a client-server architecture where the client collects facts, receives a configuration from the server, checks the current state, and runs required transactions to achieve the desired state. The Puppet server compiles configurations, acts as a file server and certificate authority, and handles reports. Puppet provides resource abstractions, templates, and modules to define reusable configurations. It aims to provide a more declarative, semantic, reproducible, and shareable approach to system administration compared to previous imperative methods.
Lutz Böhne presents Pandora's Bochs, an automated unpacker implemented using the open-source Bochs emulator. It instruments Bochs to monitor processes for behaviors indicative of unpacking, such as memory writes and branches. When such behaviors are seen, it dumps memory regions to reconstruct unpacked code and valid PE files. The system identifies processes using CR3 registers and terminates monitoring based on lack of "innovation" over time. This allows automated dynamic analysis of runtime packers without access to source code.
Coby Viner gave a technical talk at a lab meeting about GNU Parallel, a tool for executing jobs in parallel using one or more computers. Viner covered why GNU Parallel is useful, basic examples from its tutorial on syntax and usage, additional syntax for more complex tasks, and real examples of how they have used it. The talk provided an overview and introduction to GNU Parallel for parallelizing tasks, with a focus on explaining its syntax and capabilities through examples.
The document discusses translating .NET bytecode to run on the Parrot virtual machine (VM). It describes:
1) Choosing bytecode translation over other options like modifying compilers or embedding VMs, as it provides better performance and independence from programming languages.
2) The challenges of translating between the stack-based .NET bytecode and register-based Parrot bytecode.
3) The architecture developed to make the translation pluggable and declarative, including a metadata translator, instruction translator, and stack to register mapping modules.
Parrot -- "one bytecode to rule them all"Nuno Carvalho
Parrot is a virtual machine designed to compile and execute bytecode for dynamic languages. It is register-based and object-oriented. The presentation discusses Parrot's design, features like exception handling and garbage collection, and how to build compilers for Parrot using the Parrot Compiler Toolkit.
Strategies to improve embedded Linux application performance beyond ordinary ...André Oriani
he common recipe for performance improvement is to profile an application, identify the most time-consuming routines, and finally select them for optimization. Sometimes that is not enough. Developers may have to look inside the OS searching for performance improvement opportunities. Or they might need to optimize code inside a third party library they do not have access to. For those cases, other strategies shall be used. This presentation reports the experiences of Motorola's Brazilian developers reducing the startup time of an application on Motorola's MOTOMAGX embedded Linux platform. Most of the optimization was performed in the binary loading stage, prior to the execution of the entry point function. This endeavor required use of Linux ABI and Linux Loader going beyond typical bottleneck searching. The presentation will cover prelink, dynamic library loading, tuning of shared objects, and enhancing user experience. A live demo will show the use of prelink and other tools to improve performance of general Linux platforms when libraries are used.
The document provides an overview of Erlang and its features for building scalable and fault-tolerant systems. It discusses how Erlang uses lightweight processes, message passing, and supervision to allow for high concurrency. It also covers how Erlang enables hot code upgrading and distribution across nodes.
Gábor Hojtsy gave a presentation on the history and process of localization and translation for Drupal projects. He discussed how translation strings are extracted from code and stored in .po files to be shared among translators. Localize.drupal.org was created to streamline the translation process by automating tasks like parsing releases and providing a centralized web interface. The site now hosts over 5,600 translated releases and 149,000 translation strings across many projects.
This document provides an overview of Puppet, a system configuration management tool. Puppet uses a declarative language to define system resources and their desired state. It has a client-server architecture where the client collects facts, receives a configuration from the server, checks the current state, and runs required transactions to achieve the desired state. The Puppet server compiles configurations, acts as a file server and certificate authority, and handles reports. Puppet provides resource abstractions, templates, and modules to define reusable configurations. It aims to provide a more declarative, semantic, reproducible, and shareable approach to system administration compared to previous imperative methods.
Lutz Böhne presents Pandora's Bochs, an automated unpacker implemented using the open-source Bochs emulator. It instruments Bochs to monitor processes for behaviors indicative of unpacking, such as memory writes and branches. When such behaviors are seen, it dumps memory regions to reconstruct unpacked code and valid PE files. The system identifies processes using CR3 registers and terminates monitoring based on lack of "innovation" over time. This allows automated dynamic analysis of runtime packers without access to source code.
Coby Viner gave a technical talk at a lab meeting about GNU Parallel, a tool for executing jobs in parallel using one or more computers. Viner covered why GNU Parallel is useful, basic examples from its tutorial on syntax and usage, additional syntax for more complex tasks, and real examples of how they have used it. The talk provided an overview and introduction to GNU Parallel for parallelizing tasks, with a focus on explaining its syntax and capabilities through examples.
The document provides an overview of the Go programming language. It introduces the Go team members and outlines the course on Go programming. It then discusses Go's motivation as a new, experimental, concurrent, garbage-collected systems programming language. The rest of the document covers Go language basics like syntax, types, expressions, declarations and control structures.
Python for Science and Engineering: a presentation to A*STAR and the Singapor...pythoncharmers
An introduction to Python in science and engineering.
The presentation was given by Dr Edward Schofield of Python Charmers (www.pythoncharmers.com) to A*STAR and the Singapore Computational Sciences Club in June 2011.
- Erlang supports a concurrency-oriented programming model based on lightweight processes that communicate asynchronously via message passing.
- Processes are isolated from one another so that failures are contained and cannot propagate from one process to another.
- The document discusses using Erlang to build distributed, fault-tolerant systems through principles of concurrency, isolation, and pure message passing between processes.
The document discusses stacks and procedures in assembly language programming. It covers stack implementation using registers and instructions, parameter passing methods using registers or stack, and establishing stack frames using ENTER and LEAVE instructions. Procedures can be called using CALL and control returned using RET. The stack is used for temporary data storage, parameter passing, and storing return addresses for procedures.
This document discusses return-oriented programming (ROP) attacks and variants. It begins with an introduction to ROP attacks, explaining that they circumvent data execution prevention by chaining small snippets of executable code (called gadgets) that end in return instructions. It then covers different ROP attack techniques like using arithmetic, comparison, and loop gadgets to achieve Turing completeness. The document discusses challenges like handling null bytes and describes variants like jump-oriented programming (JOP) that uses indirect jumps. It also covers creating alphanumeric ROP shellcode by selecting printable addresses. In the end, it provides tips for effectively searching gadgets.
To hit Ruby3x3, we must first figure out **what** we're going to measure, **how** we're going to measure it, in order to get what we actually want. I'll cover some standard definitions of benchmarking in dynamic languages, as well as the tradeoffs that must be made when benchmarking. I'll look at some of the possible benchmarks that could be considered for Ruby 3x3, and evaluate them for what they're good for measuring, and what they're less good for measuring, in order to help the Ruby community decide what the 3x goal is going to be measured against.
Mixed-language Python/C++ debugging with Python Tools for Visual Studio- Pave...PyData
This document discusses debugging Python and C++ code together using Python Tools for Visual Studio (PTVS). Key points:
- Developers often need to debug Python code that uses C/C++ extensions or vice versa.
- PTVS allows stepping between Python and C++ code, setting breakpoints in both, and inspecting values in either language.
- However, it has limitations as the native debugger controls what the Python debugger can do. Certain Python features may not be fully supported.
This document discusses PHP and Ruby extensions that allow running Ruby scripts within PHP and PHP scripts within Ruby. It introduces php-extension ruby, which enables running Ruby code from PHP, and php_embed gem, which enables running PHP code from Ruby. It provides sample code for using these extensions and discusses challenges in implementing web applications that utilize both languages.
This document lists various promotional items like posters, pencils, stickers, and food items like cookies and granola bars that will be available in the main foyer and pool foyer on certain days of the week for students to earn signatures. Students should check the listed areas before homebase on Mondays and Wednesdays or the pool foyer on Tuesdays and Thursdays to get promotional items for collecting signatures, and should work hard to earn those signatures.
Userspace RCU library : what linear multiprocessor scalability means for your...Alexey Ivanov
RCU is well-known at the kernel-level for providing a way to synchronize shared data structures in read-often, update-rarely scenarios.
The development of a RCU library at the userspace application level has been mainly driven by the need for efficient synchronization of userspace tracing control data structures.
IBM kindly agreed to allow distribution of RCU-related code in a LGPL library, which makes it available for everyone to use. This can have large impact on the design of highly scalable applications performing caching of frequent requests, like domain name servers, proxy and web servers.
This presentation will discuss about the class of applications which could benefit from using the userspace RCU library.
The userspace RCU library is available under the LGPL license at http://www.lttng.org/urcu .
The AMD Advanced Synchronization Facility (ASF) is an experimental instruction set extension for the AMD64 architecture that would provide new capabilities for efficient synchronization of access to shared data in highly multithreaded applications as well as operating system kernels. ASF provides a means for software to inspect and update multiple shared memory locations atomically without having to rely on locks for mutual exclusion. It is intended to facilitate lock-free programming for highly concurrent shared data structures, allowing more complex and higher performance manipulation of such structures than is practical with traditional techniques based on compare-swap instructions such as CMPXCHG16B. ASF code can also interoperate with lock-based code, or with Software Transactional Memory.
Christopher Columbus was an Italian explorer who claimed land in North and South America for Spain during the Age of Exploration. Ferdinand Magellan was a Portuguese explorer who discovered an Atlantic-Pacific passage through a narrow strait, becoming the first person to circumnavigate the globe. The Columbian Exchange was the transfer of people, products, and ideas between the Western and Eastern Hemispheres following the voyages of Columbus.
The document provides definitions for 6 key vocabulary terms related to the US government: Federalism is the sharing of power between federal and state governments. John Locke was an English philosopher who believed in natural rights. Montesquieu was a French philosopher who believed in separating government powers among branches for balance. The Supreme Court is the highest US court. Checks and balances refers to the three branches having equal power to limit any one from becoming too powerful. A citizen belongs to a country and has associated rights and responsibilities.
Europeans competed for control of North America in the 17th century. Mercantilism motivated European powers like England to establish colonies to increase their wealth and power. Henry Hudson was an English explorer commissioned by the Dutch East India Company to search for a northwest passage through North America to Asia for trade. In 1609, during his fourth voyage, Hudson explored the region and discovered the river that now bears his name, though he did not complete his goal of finding the northwest passage.
My slides from "Inside PHP", a talk about how to change the syntax of the PHP programming language.
Modified PHP 5.4.4 source code (with the "until" keyword added during this presentation) is available here:
http://github.com/thomaslee/oscon2012-inside-php
The document discusses using Ragel and Lemon to generate parsers. It provides an overview of lexical analysis and syntax analysis in parsing. Ragel is introduced as a state machine generator that can be used to implement a lexer. A Ragel example tokenizes mathematical expressions. Lemon is a parser generator that outputs code to implement a parser based on a defined grammar. A Lemon example parses mathematical expressions. Finally, the document shows how to combine a Ragel lexer and Lemon parser to build a calculator application that can evaluate expressions.
Python and Perl are both high-level programming languages. Python is an interpreted language with clear, readable syntax and intuitive object orientation. Perl is also an interpreted and dynamic language that supports both procedural and object-oriented programming and is best known for powerful text processing. Both languages have a long history and are widely used for web development, scientific computing, and other tasks.
The document discusses language translation using lex and yacc tools. It begins with an introduction to compilers and interpreters. It then provides details on the phases of a compiler including lexical analysis, syntax analysis, semantic analysis, code generation, and optimization. The document also provides an overview of the lex and yacc specifications including their basic structure and how they are used together. Lex is used for lexical analysis by generating a lexical analyzer from regular expressions. Yacc is used for syntax analysis by generating a parser from a context-free grammar. These two tools work together where lex recognizes tokens that are passed to the yacc generated parser.
The document provides an overview of the Go programming language. It introduces the Go team members and outlines the course on Go programming. It then discusses Go's motivation as a new, experimental, concurrent, garbage-collected systems programming language. The rest of the document covers Go language basics like syntax, types, expressions, declarations and control structures.
Python for Science and Engineering: a presentation to A*STAR and the Singapor...pythoncharmers
An introduction to Python in science and engineering.
The presentation was given by Dr Edward Schofield of Python Charmers (www.pythoncharmers.com) to A*STAR and the Singapore Computational Sciences Club in June 2011.
- Erlang supports a concurrency-oriented programming model based on lightweight processes that communicate asynchronously via message passing.
- Processes are isolated from one another so that failures are contained and cannot propagate from one process to another.
- The document discusses using Erlang to build distributed, fault-tolerant systems through principles of concurrency, isolation, and pure message passing between processes.
The document discusses stacks and procedures in assembly language programming. It covers stack implementation using registers and instructions, parameter passing methods using registers or stack, and establishing stack frames using ENTER and LEAVE instructions. Procedures can be called using CALL and control returned using RET. The stack is used for temporary data storage, parameter passing, and storing return addresses for procedures.
This document discusses return-oriented programming (ROP) attacks and variants. It begins with an introduction to ROP attacks, explaining that they circumvent data execution prevention by chaining small snippets of executable code (called gadgets) that end in return instructions. It then covers different ROP attack techniques like using arithmetic, comparison, and loop gadgets to achieve Turing completeness. The document discusses challenges like handling null bytes and describes variants like jump-oriented programming (JOP) that uses indirect jumps. It also covers creating alphanumeric ROP shellcode by selecting printable addresses. In the end, it provides tips for effectively searching gadgets.
To hit Ruby3x3, we must first figure out **what** we're going to measure, **how** we're going to measure it, in order to get what we actually want. I'll cover some standard definitions of benchmarking in dynamic languages, as well as the tradeoffs that must be made when benchmarking. I'll look at some of the possible benchmarks that could be considered for Ruby 3x3, and evaluate them for what they're good for measuring, and what they're less good for measuring, in order to help the Ruby community decide what the 3x goal is going to be measured against.
Mixed-language Python/C++ debugging with Python Tools for Visual Studio- Pave...PyData
This document discusses debugging Python and C++ code together using Python Tools for Visual Studio (PTVS). Key points:
- Developers often need to debug Python code that uses C/C++ extensions or vice versa.
- PTVS allows stepping between Python and C++ code, setting breakpoints in both, and inspecting values in either language.
- However, it has limitations as the native debugger controls what the Python debugger can do. Certain Python features may not be fully supported.
This document discusses PHP and Ruby extensions that allow running Ruby scripts within PHP and PHP scripts within Ruby. It introduces php-extension ruby, which enables running Ruby code from PHP, and php_embed gem, which enables running PHP code from Ruby. It provides sample code for using these extensions and discusses challenges in implementing web applications that utilize both languages.
This document lists various promotional items like posters, pencils, stickers, and food items like cookies and granola bars that will be available in the main foyer and pool foyer on certain days of the week for students to earn signatures. Students should check the listed areas before homebase on Mondays and Wednesdays or the pool foyer on Tuesdays and Thursdays to get promotional items for collecting signatures, and should work hard to earn those signatures.
Userspace RCU library : what linear multiprocessor scalability means for your...Alexey Ivanov
RCU is well-known at the kernel-level for providing a way to synchronize shared data structures in read-often, update-rarely scenarios.
The development of a RCU library at the userspace application level has been mainly driven by the need for efficient synchronization of userspace tracing control data structures.
IBM kindly agreed to allow distribution of RCU-related code in a LGPL library, which makes it available for everyone to use. This can have large impact on the design of highly scalable applications performing caching of frequent requests, like domain name servers, proxy and web servers.
This presentation will discuss about the class of applications which could benefit from using the userspace RCU library.
The userspace RCU library is available under the LGPL license at http://www.lttng.org/urcu .
The AMD Advanced Synchronization Facility (ASF) is an experimental instruction set extension for the AMD64 architecture that would provide new capabilities for efficient synchronization of access to shared data in highly multithreaded applications as well as operating system kernels. ASF provides a means for software to inspect and update multiple shared memory locations atomically without having to rely on locks for mutual exclusion. It is intended to facilitate lock-free programming for highly concurrent shared data structures, allowing more complex and higher performance manipulation of such structures than is practical with traditional techniques based on compare-swap instructions such as CMPXCHG16B. ASF code can also interoperate with lock-based code, or with Software Transactional Memory.
Christopher Columbus was an Italian explorer who claimed land in North and South America for Spain during the Age of Exploration. Ferdinand Magellan was a Portuguese explorer who discovered an Atlantic-Pacific passage through a narrow strait, becoming the first person to circumnavigate the globe. The Columbian Exchange was the transfer of people, products, and ideas between the Western and Eastern Hemispheres following the voyages of Columbus.
The document provides definitions for 6 key vocabulary terms related to the US government: Federalism is the sharing of power between federal and state governments. John Locke was an English philosopher who believed in natural rights. Montesquieu was a French philosopher who believed in separating government powers among branches for balance. The Supreme Court is the highest US court. Checks and balances refers to the three branches having equal power to limit any one from becoming too powerful. A citizen belongs to a country and has associated rights and responsibilities.
Europeans competed for control of North America in the 17th century. Mercantilism motivated European powers like England to establish colonies to increase their wealth and power. Henry Hudson was an English explorer commissioned by the Dutch East India Company to search for a northwest passage through North America to Asia for trade. In 1609, during his fourth voyage, Hudson explored the region and discovered the river that now bears his name, though he did not complete his goal of finding the northwest passage.
My slides from "Inside PHP", a talk about how to change the syntax of the PHP programming language.
Modified PHP 5.4.4 source code (with the "until" keyword added during this presentation) is available here:
http://github.com/thomaslee/oscon2012-inside-php
The document discusses using Ragel and Lemon to generate parsers. It provides an overview of lexical analysis and syntax analysis in parsing. Ragel is introduced as a state machine generator that can be used to implement a lexer. A Ragel example tokenizes mathematical expressions. Lemon is a parser generator that outputs code to implement a parser based on a defined grammar. A Lemon example parses mathematical expressions. Finally, the document shows how to combine a Ragel lexer and Lemon parser to build a calculator application that can evaluate expressions.
Python and Perl are both high-level programming languages. Python is an interpreted language with clear, readable syntax and intuitive object orientation. Perl is also an interpreted and dynamic language that supports both procedural and object-oriented programming and is best known for powerful text processing. Both languages have a long history and are widely used for web development, scientific computing, and other tasks.
The document discusses language translation using lex and yacc tools. It begins with an introduction to compilers and interpreters. It then provides details on the phases of a compiler including lexical analysis, syntax analysis, semantic analysis, code generation, and optimization. The document also provides an overview of the lex and yacc specifications including their basic structure and how they are used together. Lex is used for lexical analysis by generating a lexical analyzer from regular expressions. Yacc is used for syntax analysis by generating a parser from a context-free grammar. These two tools work together where lex recognizes tokens that are passed to the yacc generated parser.
The document provides an introduction to compilers. It discusses that compilers are language translators that take source code as input and convert it to another language as output. The compilation process involves multiple phases including lexical analysis, syntax analysis, semantic analysis, code generation, and code optimization. It describes the different phases of compilation in detail and explains concepts like intermediate code representation, symbol tables, and grammars.
Open Source Compiler Construction for the JVMTom Lee
This document discusses building a compiler for a simple language called "Awesome" that targets the Java Virtual Machine (JVM). It recommends writing a stub code generator first for quick feedback before building the full compiler. The compiler will use Scala parser combinators to parse the input into an abstract syntax tree (AST) and then walk the AST to generate equivalent JVM bytecode using the Bytecode Engineering Library (BCEL). The document outlines the overall compiler architecture and next steps to expand the language features supported by the compiler.
1. The document discusses the phases of a compiler including lexical analysis, syntax analysis, and semantic analysis.
2. Lexical analysis breaks input into tokens which are passed to the parser, syntax analysis builds an abstract syntax tree by applying grammar rules to check structure, and semantic analysis ensures correct meaning.
3. Key aspects covered include context-free grammars, symbol tables for storing token information, and error detection and reporting across compiler phases.
This document provides an introduction to compiler construction. It outlines the objectives of understanding how to build a compiler, use compiler construction tools, and be familiar with concepts like virtual machines, grammars, and optimization techniques. It defines compilation as translating a program to a semantically equivalent target language and interpretation as performing operations implied by a source program. The analysis-synthesis model and how it is used in compilers and other tools is described. The phases of a compiler from preprocessing to code generation are outlined.
The document discusses various concepts related to compiler design including:
1. A compiler translates a source program into an equivalent target program and detects errors. It performs analysis and synthesis.
2. The analysis phase breaks down the source code into pieces and determines syntax and meaning. Synthesis converts the intermediate code into a target program.
3. A symbol table stores identifiers, their attributes like type, scope, and storage allocation. It also stores information about subroutines.
4. Preprocessors allow macros, include files, and perform preliminary functions before compilation.
5. Context free grammars are used to specify language syntax using terminals, non-terminals, start symbols, and productions.
Preon is a declarative data binding framework for Java that allows mapping data structures to binary encoded representations. It generates decoders, encoders, and documentation automatically based on annotations on the data structure classes. This avoids the need for manual parsing of bit streams and improves maintainability. Preon uses a chain of Codec objects to handle each encoding/decoding task, keeping the logic encapsulated and modular. It supports features like expressions, lazy loading, inheritance and custom CodecFactories to handle different data types. The goal is to hide the complexity of binary encoding and make working with binary data as easy as working with regular Java objects.
ADAM is an open source platform for scalable genomic analysis that defines a data schema, Scala API, and command line interface. It uses Apache Spark for efficient parallel and distributed processing of large genomic datasets stored in Parquet format. Key features of ADAM include its ability to perform iterative analysis on whole genome datasets while minimizing data movement through Spark. The document also describes using ADAM and PacMin for long read assembly through techniques like minhashing for fast read overlapping and building consensus sequences on read graphs.
- The document outlines the goals, outcomes, prerequisites, topics covered, and grading for a compiler design course.
- The major goals are to provide an understanding of compiler phases like scanning, parsing, semantic analysis and code generation, and have students implement parts of a compiler for a small language.
- By the end of the course students will be familiar with compiler phases and be able to define the semantic rules of a programming language.
- Prerequisites include knowledge of programming languages, algorithms, and grammar theories.
- The course covers topics like scanning, parsing, semantic analysis, code generation and optimization.
The document discusses the phases of a compiler and their functions. It describes:
1) Lexical analysis converts the source code to tokens by recognizing patterns in the input. It identifies tokens like identifiers, keywords, and numbers.
2) Syntax analysis/parsing checks that tokens are arranged according to grammar rules by constructing a parse tree.
3) Semantic analysis validates the program semantics and performs type checking using the parse tree and symbol table.
The document discusses the phases of a compiler:
1) Lexical analysis scans the source code and converts it to tokens which are passed to the syntax analyzer.
2) Syntax analysis/parsing checks the token arrangements against the language grammar and generates a parse tree.
3) Semantic analysis checks that the parse tree follows the language rules by using the syntax tree and symbol table, performing type checking.
4) Intermediate code generation represents the program for an abstract machine in a machine-independent form like 3-address code.
The document discusses the major phases of a compiler:
1. Syntax analysis parses the source code and produces an abstract syntax tree.
2. Contextual analysis checks the program for errors like type checking and scope and annotates the abstract syntax tree.
3. Code generation transforms the decorated abstract syntax tree into object code.
The document provides an overview of compiler design and the different phases involved in compiling a program. It discusses:
1) What compilers do by translating source code into machine code while hiding machine-dependent details. Compilers may generate pure machine code, augmented machine code, or virtual machine code.
2) The typical structure of a compiler which includes lexical analysis, syntactic analysis, semantic analysis, code generation, and optimization phases.
3) Lexical analysis involves scanning the source code and grouping characters into tokens. Regular expressions are used to specify patterns for tokens. Scanner generators like Lex and Flex can generate scanners from regular expression definitions.
The document introduces a convention-based syntax notation designed for both humans and parser generators. Key aspects include using conventions to simplify EBNF notation, defining rules without delimiters, inferring token sets from rules, and representing abstract syntax trees with variables. The approach was motivated by limitations of existing formalisms. Examples are provided for microsyntax, macrosyntax, and representing abstract syntax trees. Future work discussed expanding the approach to support additional language features.
Using ANTLR on real example - convert "string combined" queries into paramete...Alexey Diyan
1. Hello ANTLR: ANother Tool for Language Recognition
2. Where we can use ANTLR?
3. Why just not use regular expression language?
4. Tools under ANTLR umbrella
5. ANTLR basic syntax
6. ANTLR on real example
The document discusses lexical analysis in compilers. It begins with an overview of lexical analysis and its role as the first phase of a compiler. It describes how a lexical analyzer works by reading the source program as a stream of characters and grouping them into lexemes (tokens). Regular expressions are used to specify patterns for tokens. The document then discusses specific topics like lexical errors, input buffering techniques, specification of tokens using regular expressions and grammars, recognition of tokens using transition diagrams, and the transition diagram for identifiers and keywords.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
What is an RPA CoE? Session 1 – CoE VisionDianaGray10
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/how-axelera-ai-uses-digital-compute-in-memory-to-deliver-fast-and-energy-efficient-computer-vision-a-presentation-from-axelera-ai/
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit.
As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability and privacy. This migration, however, introduces the challenge of operating within the stringent confines of resource constraints typical at the edge, including small form factors, low energy budgets and diminished memory and computational capacities. Axelera AI addresses these challenges through an innovative approach of performing digital computations within memory itself. This technique facilitates the realization of high-performance, energy-efficient and cost-effective computer vision capabilities at the thin and thick edge, extending the frontier of what is achievable with current technologies.
In this presentation, Verhoef unveils his company’s pioneering chip technology and demonstrates its capacity to deliver exceptional frames-per-second performance across a range of standard computer vision networks typical of applications in security, surveillance and the industrial sector. This shows that advanced computer vision can be accessible and efficient, even at the very edge of our technological ecosystem.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
2. Overview
• About me!
• Implemented unified try/except/finally for Python 2.5 (Georg Brandl’s PEP 341)
• Support for compiling ASTs to bytecode from within Python programs for 2.6
• Wannabe compiler nerd.
• Terminology & brief intro to compilers:
• Grammars, Scanners & Parsers
• Parse Trees & Abstract Syntax Trees
• General architecture of a bytecode compiler
• How to add new syntax to Python:
• Python compiler architecture & summary
• Case study in adding a new keyword
• Demo!
3. “Python” vs. “CPython”
• When I say “Python” in this presentation, I mean CPython.
• Other Python implementations:
• Jython
• PyPy
• IronPython
• Stackless Python
• More...
• Many of the concepts here are likely applicable, but the details may differ wildly
• Refer to the slides for my Inside PHP presentation to see how things may differ.
• Python 3.x
4. Compilers 101: Scanners
• Or lexical analyzers, or tokenizers
• Input: raw source code
• Output: a stream of tokens
IF_TOK
IDENT("a")
if a == b
EQ_TOK
IDENT("b")
5. Compilers 101: Parsers
• Input: a stream of tokens from the scanner
• Output is implementation dependent
• The goal of a parser is to structure to the sequence of tokens from the scanner.
• Parsers are frequently automatically generated from a grammar/DSL
• See parser generators like Yacc/Bison, ANTLR, etc. or e.g. parser combinators in Haskell, Scala, ML.
IF_TOK If
Compare
IDENT("a")
Eq Name("a") Name("b")
EQ_TOK
IDENT("b")
6. Compilers 101: Context-free grammars
• Or simply “grammar”
•A grammar describes the complete syntax of a (programming) language.
• Usually expressed in Extended Backus-Naur Form (EBNF)
• Or some variant thereof.
• Variants of EBNF used for a lot of DSL-based parser generators
• e.g. Yacc/Bison, ANTLR, etc.
7. Compilers 101: Parse Tree
• Or concrete syntax tree.
• “...
an ordered, rooted tree that represents the [full] syntactic structure of a string
according to some formal grammar.”
• http://en.wikipedia.org/wiki/Parse_tree
• Python’s parser constructs a parse tree based on Grammar/Grammar
• Must be manually converted to Abstract Syntax Tree before we do anything fun.
8. Compilers 101: Abstract Syntax Tree
• AST
• Rooted tree data structure.
• In-memory, intermediate representation of the input (source) program.
• “Abstract” in that it doesn’t necessarily include every detail that appears in the real syntax.
•A code generator can traverse an AST and emit target code.
• e.g. Python byte code
If
• Certain
optimizations are also convenient at
the AST level. Compare
Eq Name("a") Name("b")
10. Generalized *Python* Compiler Architecture
Source files Source code Scanner Token stream
rammar*
Grammar/G
* Grammar/Grammar is processed by Parser/pgen to generate a scanner & parser
r*
/Gramma Parser
Grammar
&
.c
P ython/ast
Bytecode Abstract
Bytecode Code Generator
Interpreter Syntax Tree
arser/Py thon.asdl
n/ceval.c Python/compile.c P
Pytho
11. Case Study: The “unless” statement
It’s basically
will_give_you_up = False
“if not ...”
unless will_give_you_up:
print(“Never gonna give you up.”)
will_let_you_down = True
unless will_let_you_down:
print(“Letting you down :’(”)
-- output --
Never gonna give you up.
12. Adding the “unless” construct to (C)Python
1.Describe the syntax of the new construct
2.Add new AST node(s) if necessary
3.Write a parse tree to Abstract Syntax Tree (AST) transform
4.Emit bytecode for your new AST node(s)
13. Before you start...
• You’ll need the usual gcc toolchain, GNU Bison, etc.
• Debian/Ubuntuapt-get install build-essential
• OSX Xcode command line tools should give you most of what you need.
• Grab the Python code from Mercurial
• I’m working with the stable Python 3.2 branch
• Only because tip was breaking in weird & wonderful ways at the time of rehearsal!
• The steps described here should also work for any version of Python >= 2.5.x
• Beware of cyclic dependencies in the build process when modifying the grammar.
14. 1. Describe the syntax of the new construct
• Or: describe how your new language construct “looks” in source code
• Modify the grammar in Grammar/Grammar
• EBNF-ish DSL Scanner Token stream
• Used by Parser/pgen, Python’s custom
parser generator, to generate the C code that rammar*
Grammar/G
drives Python’s parser and scanner.
rammar* Parser
Grammar/G
• Riffing
on existing constructs is a great way to learn: &
.c
P ython/ast
how does an if statement look like in the grammar?
• Let’s specify the syntax of unless
15. 2. Add new AST node(s) if necessary
• Parser/Python.asdl
• Zephyr Abstract Syntax Definition Language -- a DSL for describing ASTs.
• http://asdl.sourceforge.net/
• The Python build process generates C code describing the AST.
• See Parser/asdl_c.py and Parser/asdl.py for details of how this code is generated.
• Can you represent your new Python construct using existing AST nodes?
• Great! Skip this step.
• Let’s go ahead and add an Unless node anyway.
• Demonstration using If and Not nodes if we have time ... If
Compare
Eq Name("a") Name("b")
16. 3. Parse tree to AST transformation
• Recall the parser that is generated from Grammar/Grammar
• The generated parser constructs a parse tree without any effort on our part ...
• ...
BUT! earlier we saw that the
*
Grammar
code generator takes an AST as input. / Parser
Grammar
&
.c
P ython/ast
• We need to manually transform the parse See?
tree into an AST.
Abstract
Code Generator
• Python/ast.c
Syntax Tree
l
pile .c Parser/P ython.asd
Py thon/com
• Again,look to the existing if statement for If
Compare
inspiration. Eq Name("a") Name("a")
17. 4. Emit bytecode for the new AST node
• This step is only necessary if you added a new AST node in step 2.
• At this point our AST is ready to compile to bytecode.
• Traverse the tree, emitting bytecode as we go.
• What bytecode instructions do we need to use?
• What does the bytecode for a simple if statement look like? How would we modify it?
18. Intermission: Python VM + bytecode
data = []
• Python has a stack based virtual machine. def push(x):
data.append(x)
• Values get pushed onto a data stack.
def pop():
• Opcodes manipulate the stack. return data.pop()
• e.g. binary op: pop two values, compute result, push new value.
def add():
• Control flow using unconditional & conditional jumps. push(pop() + pop())
def sub():
push(pop() - pop())
push(5) # [5]
push(6) # [5, 6]
add() # [11]
19. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
20. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
21. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
22. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
23. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
24. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
25. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
26. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
27. Bytecode for “if x == y: ...”
LOAD_NAME x
LOAD_NAME y x == y
COMPARE_OP PyCmp_EQ
POP_JUMP_IF_FALSE end
... if …: ...
...
... end
28. 4. Emit bytecode for the new AST node (cont.)
• Python/compile.c
• Look for <AST>_kind to track down existing impls for AST nodes -- e.g. ClassDef_kind for classes
• Once again: how is if implemented?
• Look for If_kind
• How can we modify this to implement unless?
29. Demo!
• The usual ./configure && make dance on Linux/OSX.
• With a little luck, magic happens and you get a binary in the root directory.
• ... So, uh, does it work?
30. And exhale.
• Lots to take in, right?
• In my experience, this stuff is best learned bit-by-bit through practice.
• Ask questions!
• Google
• Python-Dev
• Or hey, ask me...