Python Intro For Managers

Python: a Powerful, Easy-to-Use, Open-Source Scripting Language Stephen Ferg Bureau of Labor Statistics

What we're going to talk about...

Open-source applications -- or at least some of them -- are good.

Scripting languages are good.

Python is the best scripting language.

Executive Summary Time, money, and innovative energy... can be saved. Productivity, speed, and quality ... can be improved.

Topics

Scripting languages

Python

Issues surrounding use of Python

Open-Source Software

What the experts think

Where we might find Python useful

What is a "scripting" language?

Interpreted

requires a run-time interpreter or virtual machine

Untyped or dynamically typed

No data declarations

No compilation step

In the beginning...

System programming languages

Assembler, C

Fortran, Cobol, Algol

PL/1, Pascal, Basic, C++, Java

Command languages

JCL

TSO CLists, CMS "execs"

Batch files

"Little Languages" appear...

awk

sed

grep

Scripting languages evolve

UNIX shell scripting languages

Rexx

Tcl, TK

Perl

Python

PHP

Ruby

SAS

Many have higher-level object-oriented features that make them powerful application development languages in their own right.

The defining characteristic of scripting languages...

Vastly increased productivity!

"Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout

IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html

Assembly Language

One machine instruction per line

System programming languages

3-7 machine instructions per line

Scripting languages

Hundreds to thousands of instructions per line

Programmers can write roughly the same number of lines of code per year regardless of language. Productivity = the number of machine instructions that a programmer can produce per year.

None Strong Degree of Typing Assembler System Languages Scripting Languages VB Python, Perl, Ruby, TCL C C++ Java Instructions/Statement 1000 100 10 1 Language Levels and Productivity From "Scripting: Higher Level Programming for the 21st Century" by John K. Ousterhout. This version prepared by Dana Moore and updated by Stephen Ferg

This leads to... increasing use of scripting languages.

John Ousterhout

Scripting: Higher Level Programmingfor the 21st Century - IEEE Computer 1998

Scripting languages represent a different set of tradeoffs than system programming languages. They give up execution speed and strong typing but provide significantly higher programmer productivity and software reuse. This tradeoff makes increasing sense as computers become faster and cheaper compared to programmers. For the last fifteen years a fundamental change has been occurring in the way people write computer programs. ...from system programming languages to scripting languages. This article explains why scripting languages will handle many of the programming tasks of the next century better than system programming languages.

Robert C. Martin

I think there is a trend in language that will become more and more evident as the decade progresses. I think we are seeing an end to the emphasis on statically typed languages like C++, Java, Eiffel, Pascal, and Ada.

I expect to see an ever increasing use of dynamically typed languages, such as Python, Ruby, and even Smalltalk. These languages, and languages of their kind, will be mainstream industrial languages in the coming years.

Tim O'Reilly

People are so stuck in the personal computer paradigm that they don't recognize that the nature of applications has undergone a profound change in the last decade, with most of the new killer apps running on what has been called the LAMP platform (Linux-Apache-MySQL-PHP | Perl | Python). People understand the importance of Linux, Apache and MySQL... but they still struggle with understanding the "P" in LAMP.

The reason why dynamic languages like Perl, Python, and PHP are so important is key to understanding the paradigm shift. Unlike applications from the previous paradigm, web applications are not released in one to three year cycles. They are updated every day, sometimes every hour.

Why Scripting Languages Matter

Agile programming languages

We really should stop

calling them

"scripting" languages.

"Agile" languages

would be more accurate.

Kevin Altis and Ward Cunningham

Python - a great agile programming language!

Powerful

Easy-to-learn

Easy-to-use

Open-Source

"Python" from

"Monty Python's Flying Circus"

Python language features

Derived from ABC, Modula-3, and C

Object-Oriented

Dynamically typed

Interpreted

Cross-platform (Unix, Windows, etc.)

Extensible

Flexible

Python's most obvious feature

Uses indentation as a control structure

no DO.. END no BEGIN..END no { .. }

Indentation as a control-structure

for i in range(20):

if i%3 == 0:

print i

if i%5 == 0:

print "Bingo!"

print "---"

0 Bingo! --- --- --- 3 --- --- --- 6 --- --- --- 9 --- --- --- 12 --- --- --- 15 Bingo! --- --- --- 18 --- ---

Sample Python Code

See the handouts distributed with this presentation.

For a quick overview of Python's features: http://www.ferg.org/python_slides/index.html

Python's advantages

Productivity and Ease-Of-Use

Maintainability

Flexibility (OO & functional programming)

Power

Plays well with other languages

Jython compiles to Java byte code

Extensible

Easy to extend with, or call from, C

Python – some history

Developed by

Guido van Rossum

in 1991.

A fan of Monty Python's Flying Circus

Guido van Rossum

Python BDFL - Benevolent Dictator for Life

1999 - Dr. Dobb's Journal Excellence in Programming Awards

2002 – Free Software Foundation (FSF) Award for the Advancement of Free Software

"Doctor Fun has the dubious distinction of being the first web cartoon. Doctor Fun was not, however, the first cartoon on the Internet." - http:// www.ibiblio.org/Dave/index.html

T-shirt Slogan

Python:

Programming the way Guido indented it

Who is using Python? What are they doing with it?

Industrial Light & Magic , maker of the Star Wars films, uses Python extensively in the computer graphics production process.

Disney Feature Length Animation uses Python for its animation production applications.

Google , a leading internet search engine, is powered by Python.

Yahoo uses Python for its groups site.

The Inktomi (formerly Infoseek, now part of Yahoo) search engine uses Python.

IBM and Philips have used Python to create the business practice logic for factory tool control applications.

NASA uses Python in several large projects, including a CAD/CAM system and a graphical workflow modeler used in planning space shuttle missions.

The National Institutes of Health (USA) and Case Western Reserve University are building cutting-edge genetic analysis software with Python.

The National Weather Service (USA) uses Python to prepare weather forecasts.

Python is also used for this purpose at the Swedish Meteorological and Hydrological Institute and at TV4 Sweden .

Chandler , the new open-source cross-platform Personal Information Manager being developed by Mitch Kapor, is being written in Python and wxWindows.

Lawrence Livermore National Laboratories is basing a new numerical engineering environment on Python.

The Theoretical Physics Division at Los Alamos National Laboratory uses Python to control large-scale physics codes on massively parallel supercomputers, high-end servers, and clusters.

US Navy uses Python & Zope for a web based workflow system

US Dept. of Agriculture - Python & Zope for massive collaboration

Should we be using Python? ....

Issues to Consider when Evaluating a Programming Language Let's look at some...

Are capabilities an issue?

"Batteries included" philosophy

Standard distribution includes extensive module library

Many other modules available

Frank Stajano

The Python Standard Library

GUI

strings

regular expressions

database connectivity

HTTP, CGI, HTML, XML

numeric processing

debugger

object persistence

Is execution speed an issue?

Modern processors generally make language speed a non-issue

Many applications are limited by speed of database or network connection, not programming language

Ease-of-use makes implementing optimization algorithms easier – possible to beat even C programs

Easy to write interface to C extension modules for optimization

Probably 10 times slower than a system language, but ...

"In terms of run time and memory consumption, scripting languages often turn out better than Java and not much worse than C or C++."

An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program University of Karlsruhe, Germany Technical Report 2000-5, March 10, 2000 http:// wwwipd.ira.uka.de/~prechelt/Biblio/jccpprtTR.pdf

"It might seem that the typeless nature of scripting languages could allow errors to go undetected, but in practice scripting languages are just as safe as system programming languages."

"Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout

IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html

Is dynamic typing an issue?

"I'd been a statically typed bigot for quite a few years. Four years ago I got involved with Extreme Programming. ... I liked the emphasis it put on testing.

About two years ago I noticed I was depending less and less on the type system for safety. My unit tests were preventing me from making type errors.

So I tried writing some applications in Python, and then Ruby. I found that type issues simply never arose. My unit tests kept my code on the straight and narrow. I simply didn't need static type checking."

Robert C. Martin

Is productivity an issue?

"5-10 times productivity (really!)"

Bruce Eckel

" I find that I'm able to program about three times faster [in Python] than I could in Java, and I was able to program in Java about three times faster than I could in C ."

Andy Hertzfeld

"The results indicate that, for the given programming problem, 'scripting languages' (Perl, Python, Rexx, Tcl) are more productive than conventional languages."

University of Karlsruhe, Germany, Technical Report 2000-5, March 2000

When a 20,000 line project went to approximately 3,000 lines overnight, and came out being more flexible and robust ... I realized I was on to something really good.

-- Matthew "Glyph" Lefkowitz

...the lines of Python code were 10% of the equivalent C++ code.

-- Greg Stein , eShop

"Programming is fun again!"

Over and over on comp.lang.python there are messages:

"Now that I've discovered Python, I enjoy programming again!"

"Now I am back programming in Java because the projects I'm working on call for it. But I wish I was programming in Ruby or Python ..."

Robert C. Martin

Is maintainability an issue?

"I realized that the flexibility of dynamically typed languages makes writing code significantly easier. Modules are easier to write, and easier to change."

Robert C. Martin -- http://www.artima.com/weblogs/viewpost.jsp?thread=4639

"Python excels at rapid creation of maintainable code"

Bruce Eckel

Is weirdness an issue?

"Python's use of whitespace stopped feeling unnatural after about twenty minutes.

I just indented code, pretty much as I would have done in a C program anyway, and it worked."

Eric S. Raymond

How weird is it, really?

"Most people who use Python consider the indentation syntax to be an important, if not downright critical, feature of the language. It forces you to write readable code, which in turn fosters code maintainability. It's a big win, once you get past the initial shock.

In any structured programming language, the indentation of blocks really does mean something. Most Python users think that enforcing consistency in indentation is not only good software engineering, it's simple common sense. The end result is code that is so well laid out that it resembles something akin to poetry."

Mark Lutz, author of Programming Python

Is support an issue?

Python is an "open-source" language.

It has no vendor.

Does that mean we'll have support problems?

What about...

Vendor longevity?

Consulting & training support?

Books and reference materials?

Tools? IDEs, debuggers, screen-painters?

What is "Open-Source"?

a distribution license for source code

source code is available without $$$ charge

code may be changed, customized, enhanced

GPL – Gnu Public License

Python license – unlike the GPL, you may distribute a modified version without making your changes open source.

a development style and a culture...

The Cathedral & the Bazaar

Linux is subversive. Who would have thought ... that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet?

Certainly not I... I believed there was a certain critical complexity above which a more centralized, a priori approach was required. ... the most important software needed to be built like cathedrals, carefully crafted by individual wizards or small bands of mages working in splendid isolation, with no beta to be released before its time.

The Cathedral & the Bazaar

The Linux style of development came as a surprise. No quiet, reverent cathedral-building here—rather, the Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches out of which a coherent and stable system could seemingly emerge only by a succession of miracles.

The Linux world not only didn't fly apart in confusion -- it seemed to go from strength to strength at a speed barely imaginable to cathedral-builders.

Some open-source products

Linux

Apache

MySql

PHP | Perl | Python

Apache has overwhelmingly dominated the Web server market since 1996. PHP is the most popular Apache module, running on almost 10 million domains (over a million IP addresses). "MySQL threatens to do for databases what Linux has done for operating systems." – Tim O'Reilly "LAMP"

Is Open-Source software used in the Federal Government?

See earlier list of Python users

NIH, NASA, Navy, Agriculture, Weather Service

In 2002, a Mitre study found 115 FOSS products in use in DoD

http://egovos.org/pdf/dodfoss.pdf

Why would a Federal agency use open-source software? ...

Government Computer News November 20, 2000

The NASA Acquisition Internet Service (NAIS) development team adopted open-source software several years ago and we plan to expand its use in the agency-wide procurement system.

We were using a proprietary Web development application that promised interoperability with another vendor’s database software. It failed to interoperate, however... Then we discovered Perl and have been using it for the last five years to develop and support all NAIS applications. Recently, price restructuring for a commercial DBMS threatened to consume most of the NAIS budget. We decided to convert NAIS to MySQL. Our tests showed MySQL could perform NAIS functions faster. Cost of the optional technical support was about 1 percent of that for the commercial product.

Technical support for MySQL has been excellent when we needed it, plus there are hundreds of Web sites that offer free help and support for such open-source products. We plan to evaluate the Apache HTTP Server to correct limitations of the commercial Web server we currently use.

eGov & Open-Source

Center of Open Source & Government

( http://egovos.org/index.html )

EGOVOS - high-level international conference on OSS ("Libre Software"), interoperability and open standards in government

October 2002 & March 2003 - Washington, DC

EGOVOS3 - 24-26 November, 2003 at UNESCO headquarters in Paris

The Open Source Reference Book 2003 - What Local/National Governments, the Defense Establishment, and The Global 1000 Need To Know About Open Source Software ( November 2003)

... will provide a Generally Regarded As Safe (GRAS) list of Open Source software to identify mature and useable Open Source projects

... will list Open Source software that is NIAP* or Common Criteria evaluated

*NIAP: National Information Assurance Partnership – NIST security certification

So... Is Open-Source Safe ?

Vendors and products vary widely in both the commercial and open-source arena.

The fact that a piece of software is commercial is no guarantee of its quality, or of its vendor's long-term survival.

The best open-source software is as good as the best commercial software.

Each product and vendor should be evaluated on its own merits, regardless of whether it is commercial or open-source.

Python is in the same league as the best software anywhere, commercial or open-source.

The Bottom Line

Is vendor longevity an issue?

What if Guido got run over by a bus?

The Python Software Foundation

A non-profit organization for advancing open-source technology related to Python

Holds Python's intellectual property rights.

Produces the core Python distribution, available to the public free of charge.

Establishes PSF licenses, ensuring the rights of the public to freely obtain, use, redistribute, and modify intellectual property held by the PSF.

Is mindshare an issue?

International Python Conference

IPC - in USA since 1992

EuroPython conference

in Europe since 2002

Python for Scientific Computing Workshop

SciPy - in USA since 2002

The Python community is very active and growing rapidly

Newsgroup Activity comp.lang.* December 2002

java 26953

c++ 19913

c 13874

perl 10486

python 9647

basic 7909

ruby 6466

lisp 6132

tcl 5256

pascal 4229

smalltalk 2398

fortran 2355

cobol 1845

Statistics compiled by Aaron K. Johnson.

TIOBE Popularity of Programming Languages Index July 2003 based on the number of hits returned by a Google search

1 Java 44.3

2 C 36.8

3 C++ 33.2

4 Perl 18.3

5 (Visual) Basic 15.5

6 PHP 7.6

7 SQL 6.0

8 C# 3.5

9 JavaScript 3.3

10 Delphi/Pascal/Kylix 3.1

11 Python 2.6

12 COBOL 2.3

13 SAS 2.2

14 Fortran 1.9

Index is available at http://www.tiobe.com/tpci.htm

Is online support an issue?

comp.lang.python -- Outstanding!!

http://groups.google.com/groups

?&group=comp.lang.python

Consulting and Training Resources?

Not much!

Python is probably too easy-to-learn and easy-to-use to support much of a training/ consulting industry. You can learn it out of a book!

A couple of useful consulting resources...

Zope Corp.

Fourthought , Inc. - XML tools for Python and XML and web-based applications.

Is Ease-of-Learning an issue?

Python is famously easy to use and easy to learn.

I talked my colleagues into using Python for our Computer Science 1 course this fall. ... In the past I would be swamped during office hours with students wanting help deciphering C++ compiler errors. This semester almost nobody has stopped by for syntax issues.

-- Dave Reed on Python In Education mailing list

Books & Reference Materials?

Online Materials?

Python distribution includes:

Tutorial, Language Reference

Extensive Standard Library documentation

"How to Think Like a Computer Scientist with Python" http://greenteapress.com/thinkpython/

"Python Programming – an Introduction to Computer Science" http://mcsp.wartburg.edu/zelle/python/

"Dive Into Python" http://diveintopython.org/index.html

Too many others to list...

Tools? - IDEs

IDLE comes with Python

WingIDE – excellent IDE with visual debugger

$35 and $180 -- http://wingide.com/

Visual Python

Python plug-in for Visual Studio .NET.

Python-specific features within the familiar Visual Studio environment.

Visual Python integrates seamlessly with Visual Studio .NET, allowing programmers to leverage features of Microsoft's popular development tool suite.

http://www.activestate.com/Products/Visual_Python

Tools? – Screen Painters

A screenshot of wxDesigner

What Do the Experts Think of Python?

Bruce Eckel

His book Thinking in C++ was given the Software Development Jolt Award for best book published in 1995.

Thinking in Java received Java World Reader's Choice Award and Java World Editor's Choice Award for best book, the Java Developer's Journal Editor's Choice Award for books, the Software Development Productivity Award in 1999, the third edition received the Software Development Magazine Jolt award for best technical book, 2002. One of "the industry's leading lights" ( Windows Tech Journal , September 1996).

The language you speak affects what you can think. "Python fits my brain."

Python excels at rapid creation of maintainable code

Programmer productivity is the most important thing. 5-10 times productivity (really!)

Simplicity really does make a difference.

I can remember many Python idioms because they’re simpler.

One more reason I program faster in Python.

I still have to look up how to open a file every time I do it in Java.

Python & “The Tipping Point”

It is possible to write programs to automate every task. But you don’t.

Python makes it easy enough

Eric S. Raymond

The Cathedral and the Bazaar

www.catb.org/~esr/writings/cathedral -bazaar/cathedral-bazaar/

The New Hacker's Dictionary

http://www.jargon.8hz.com/jargon_toc.html

Homesteading the Noosphere

http://www.firstmonday.dk/issues/issue3_10/raymond/

Well-known Unix guru, Linux advocate, and author

This could be an opportunity to get some hands-on experience with Python...

I charged ahead and roughed out some code ...

http://pythonology.org/success&story=esr Why Python?

I noticed I was generating working code nearly as fast as I could type. When you're writing working code nearly as fast as you can type, it generally means you've achieved mastery of the language. But that didn't make sense, because it was still day one ...

This was my first clue that, in Python, I was actually dealing with an exceptionally good design.

Not that it took me very long to learn the feature set.

This reflects another useful property of Python: it is compact -- you can hold its entire feature set (and at least a concept index of its libraries) in your head.

The long-term usefulness of a language comes from how well and how unobtrusively it supports the day-to-day work of programming, which consists not of writing new programs, but mostly reading and modifying existing ones.

So the real punchline of the story is this: weeks and months after writing fetchmailconf [my Python program], I could still read the code and grok what it was doing without serious mental effort.

Martin C. Brown

Author and Perl expert

Perl: The Complete Reference

Perl Annotated Archives

... and ... ... and ...

Nicholas Petreley ComputerWorld columnist

One of my favorite programming languages is Python. It seems I don't go a week these days without someone asking me what I know about Python, so it seems to be gaining quite a following in mainstream IT.

November, 2002

The Bottom Line...

"Use the Best Tool for the Job: Put Both a Scripting and Systems Language in Your Toolbox"- Bill Venners

http://www.artima.com/commentary/langtool.html

Python would be a useful tool in the toolboxes of our developers, DBAs, and LAN administrators,

for situations where....

A command-language is too under-powered, and a systems programming language would be overkill.

Speed and minimizing effort are important

One-time, throw-away programs

Internal utilities

Prototyping

Test scaffolding

Cross-platform portability is important

System administrators need learn only one scripting language

Prototype/develop on one platform, deploy on another (e.g. Windows NT and Unix)

Readability & maintainability are important

XML processing

The Python people also piped up to say “everything's just fine here” but then they always do. I really must learn that language.

XML Is Too Hard For Programmers

Tim Bray, co-author of the original XML 1.0 spec

Ease-of-learning is important

An application is written in four different languages (Java, C, Perl, and Unix shell-script) because it was built by four different developers who were expert in four different languages. Everybody knows this is a problem, but nobody has time to learn another language. One solution -- a single common language that is both powerful enough to handle a wide variety of tasks, and easy enough to learn quickly and easily.

More Online Information