This document provides an introduction to Python including:
- The major versions of Python and their differences
- Popular integrated development environments for Python
- How to set up Python environments using Anaconda and Eclipse
- An overview of Python basics like variables, numbers, strings, lists, dictionaries, modules and functions
- Examples of Python control flow structures like conditionals and loops
Python is an interpreted, high-level, and general-purpose programming language.
It is an easy to learn general-purpose programing language.
It is a platform-independent programing language, which means it can be used on any machine and in any operating system.
It has a simple syntax
Python is a case sensitive language.
It is an interrupted language.
It is free to use and even free for commercial products.
In this chapter we will explore strings. We are going to explain how they are implemented in Java and in what way we can process text content. Additionally, we will go through different methods for manipulating a text: we will learn how to compare strings, how to search for substrings, how to extract substrings upon previously settled parameters and last but not least how to split a string by separator chars. We will demonstrate how to correctly build strings with the StringBuilder class. We will provide a short but very useful information for the most commonly used regular expressions.
learn how to program with python
from scratch to be an expert
use the link in the first slide to get the full course
here is the link if you want : https://oke.io/JdxdUl
the link of the documentation : https://oke.io/J6DeY3
link to download python : https://oke.io/BENgRLR2
Python is an interpreted, high-level, and general-purpose programming language.
It is an easy to learn general-purpose programing language.
It is a platform-independent programing language, which means it can be used on any machine and in any operating system.
It has a simple syntax
Python is a case sensitive language.
It is an interrupted language.
It is free to use and even free for commercial products.
In this chapter we will explore strings. We are going to explain how they are implemented in Java and in what way we can process text content. Additionally, we will go through different methods for manipulating a text: we will learn how to compare strings, how to search for substrings, how to extract substrings upon previously settled parameters and last but not least how to split a string by separator chars. We will demonstrate how to correctly build strings with the StringBuilder class. We will provide a short but very useful information for the most commonly used regular expressions.
learn how to program with python
from scratch to be an expert
use the link in the first slide to get the full course
here is the link if you want : https://oke.io/JdxdUl
the link of the documentation : https://oke.io/J6DeY3
link to download python : https://oke.io/BENgRLR2
Python- Creating Dictionary,
Accessing and Modifying key: value Pairs in Dictionaries
Built-In Functions used on Dictionaries,
Dictionary Methods
Removing items from dictionary
This presentation was given online in July 2017 and will be given at the NY Java SIG later this year. It progressively builds on Java 8 concepts using puzzles and coding to give students confidence in their Java 8 stream/lambda skills. Handouts and code in https://github.com/boyarsky/java-8-streams-by-puzzles
Class 12 Computer Science, Chapter 4 - Using Python Libraries. Self learning Presentation in the form of Teacher - Student conversation.
Size 20.1 MB ppt format is also available at the same site Size 5.4 MB
In this chapter we will learn about arrays as a way to work with sequences of elements of the same type. We will explain what arrays are, how we declare, create, instantiate and use them. We will examine one-dimensional and multidimensional arrays. We will learn different ways to iterate through the array, read from the standard input and write to the standard output. We will give many example exercises, which can be solved using arrays and we will show how useful they really are.
Learn how to use lists in Java, how to use List<T> and ArrayList<T>, how to process lists of elements.
Watch the video lesson and access the hands-on exercises here: https://softuni.org/code-lessons/java-foundations-certification-lists
The basics of Python are rather straightforward. In a few minutes you can learn most of the syntax. There are some gotchas along the way that might appear tricky. This talk is meant to bring programmers up to speed with Python. They should be able to read and write Python.
Learn how to use arrays in Java, how to enter array, how to traverse an array, how to print array and more array operations.
Watch the video lesson and access the hands-on exercises here: https://softuni.org/code-lessons/java-foundations-certification-arrays
A tour of Python: slides from presentation given in 2012.
[Some slides are not properly rendered in SlideShare: the original is still available at http://www.aleksa.org/2015/04/python-presentation_7.html.]
Python- Creating Dictionary,
Accessing and Modifying key: value Pairs in Dictionaries
Built-In Functions used on Dictionaries,
Dictionary Methods
Removing items from dictionary
This presentation was given online in July 2017 and will be given at the NY Java SIG later this year. It progressively builds on Java 8 concepts using puzzles and coding to give students confidence in their Java 8 stream/lambda skills. Handouts and code in https://github.com/boyarsky/java-8-streams-by-puzzles
Class 12 Computer Science, Chapter 4 - Using Python Libraries. Self learning Presentation in the form of Teacher - Student conversation.
Size 20.1 MB ppt format is also available at the same site Size 5.4 MB
In this chapter we will learn about arrays as a way to work with sequences of elements of the same type. We will explain what arrays are, how we declare, create, instantiate and use them. We will examine one-dimensional and multidimensional arrays. We will learn different ways to iterate through the array, read from the standard input and write to the standard output. We will give many example exercises, which can be solved using arrays and we will show how useful they really are.
Learn how to use lists in Java, how to use List<T> and ArrayList<T>, how to process lists of elements.
Watch the video lesson and access the hands-on exercises here: https://softuni.org/code-lessons/java-foundations-certification-lists
The basics of Python are rather straightforward. In a few minutes you can learn most of the syntax. There are some gotchas along the way that might appear tricky. This talk is meant to bring programmers up to speed with Python. They should be able to read and write Python.
Learn how to use arrays in Java, how to enter array, how to traverse an array, how to print array and more array operations.
Watch the video lesson and access the hands-on exercises here: https://softuni.org/code-lessons/java-foundations-certification-arrays
A tour of Python: slides from presentation given in 2012.
[Some slides are not properly rendered in SlideShare: the original is still available at http://www.aleksa.org/2015/04/python-presentation_7.html.]
Python is an interpreted high-level general-purpose programming language. Its design philosophy emphasizes code readability with its use of significant indentation. Its language constructs as well as its object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects
Raspberry Pi - Lecture 5 Python for Raspberry PiMohamed Abdallah
Python syntax, Strings and console output, Conditional an, control flow, Functions, Lists and dictionaries, Loops, Bitwise operators, Classes, File Input/output
Chapter 22. Lambda Expressions and LINQIntro C# Book
In this chapter we will become acquainted with some of the advanced capabilities of the C# language. To be more specific, we will pay attention on how to make queries to collections, using lambda expressions and LINQ, and how to add functionality to already created classes, using extension methods. We will get to know the anonymous types, describe their usage briefly and discuss lambda expressions and show in practice how most of the built-in lambda functions work. Afterwards, we will pay more attention to the LINQ syntax – we will learn what it is, how it works and what queries we can build with it. In the end, we will get to know the meaning of the keywords in LINQ, and demonstrate their capabilities with lots of examples.
Very quick introduction to the language R. It talks about basic data structures, data manipulation steps, plots, control structures etc. Enough material to get you started in R.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
3. 4 Major Versions of Python
»“Python” is written in C/C++
- Version 2.7 came out in mid-2010
- Version 3.1.2 came out in early 2010
»“Jython” is written in Java for the JVM
»“IronPython” is (was!) written in C# for the .Net environment
8. Python Interactive Shell
% python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
You can type things directly into a running Python session
>>> 2+3*4
14
>>> name = "Andrew"
>>> name
'Andrew'
>>> print "Hello", name
Hello Andrew
>>>
9. The Python Interpreter
• Python is an interpreted language
• The interpreter provides an
interactive environment to play
with the language
• Results of expressions are printed
on the screen
>>> 3 + 7
10
>>> 3 < 15
True
>>> 'print me'
'print me'
>>> print 'print me'
print me
>>>
10. The print Statement
>>> print 'hello'
hello
>>> print 'hello', 'there'
hello there
• Elements separated by commas
print with a space between them
• A comma at the end of the
statement (print ‘hello’,) will not
print a newline character
11. No Braces, only Spaces!
»Python uses indentation
instead of braces to
determine the scope of
expressions
»All lines must be indented
the same amount to be part
of the scope (or indented
more if part of an inner
scope)
»This forces the programmer to
use proper indentation since
the indenting is part of the
program!
12. Variables
»Are not declared, just assigned
»The variable is created the first time you assign it
a value
»Are references to objects
»Type information is with the object, not the
reference
»Everything in Python is an object
15. Numbers: Integers
»Integer – the equivalent of a C
long
»Long Integer – an unbounded
integer value.
>>> 132224
132224
>>> 132323 ** 2
17509376329L
>>>
16. Numbers: Floating Point
»int(x) converts x to an integer
»float(x) converts x to a floating point
»The interpreter shows
a lot of digits
>>> 1.23232
1.2323200000000001
>>> print 1.23232
1.23232
>>> 1.3E7
13000000.0
>>> int(2.0)
2
>>> float(2)
2.0
18. Basic operations
»Assignment:
size = 40
a = b = c = 3
»Numbers
integer, float
complex numbers: 1j+3, abs(z)
»Strings
'hello world', 'it's hot'
"bye world"
continuation via or use """ long text """"
19. Date
» import datetime
» now = datetime.datetime.now()
» print
» print "Current date and time using str method of datetime object:"
» print str(now)
» print
» print "Current date and time using instance attributes:"
» print "Current year: %d" % now.year
» print "Current month: %d" % now.month
» print "Current day: %d" % now.day
» print "Current hour: %d" % now.hour
» print "Current minute: %d" % now.minute
» print "Current second: %d" % now.second
» print "Current microsecond: %d" % now.microsecond
» print
» print "Current date and time using strftime:"
» print now.strftime("%Y-%m-%d %H:%M")
20. Lists
»lists can be heterogeneous
a = ['spam', 'eggs', 100, 1234, 2*2]
»Lists can be indexed and sliced:
a[0] spam
a[:2] ['spam', 'eggs']
»Lists can be manipulated
a[2] = a[2] + 23
a[0:2] = [1,12]
a[0:0] = []
len(a) 5
21. List methods
»append(x)
»extend(L)
append all items in list (like Tcl lappend)
»insert(i,x)
»remove(x)
»pop([i]), pop()
create stack (FIFO), or queue (LIFO) pop(0)
»index(x)
return the index for value x
22. List methods
»count(x)
how many times x appears in list
»sort()
sort items in place
»reverse()
reverse list
23. String Literals
»Strings are immutable
»There is no char type like
in C++ or Java
»+ is overloaded to do
concatenation
>>> x = 'hello'
>>> x = x + ' there'
>>> x
'hello there'
24. Strings share many features with lists
>>> smiles = "C(=N)(N)N.C(=O)(O)O"
>>> smiles[0]
'C'
>>> smiles[1]
'('
>>> smiles[-1]
'O'
>>> smiles[1:5]
'(=N)'
>>> smiles[10:-4]
'C(=O)'
Use “slice” notation to
get a substring
25. String operations
»concatenate with + or neighbors
word = 'Help' + x
word = 'Help' 'a'
»subscripting of strings
'Hello'[2] 'l'
slice: 'Hello'[1:2] 'el'
word[-1] last character
len(word) 5
immutable: cannot assign to subscript
26. String Methods: find, split
smiles = "C(=N)(N)N.C(=O)(O)O"
>>> smiles.find("(O)")
15
>>> smiles.find(".")
9
>>> smiles.find(".", 10)
-1
>>> smiles.split(".")
['C(=N)(N)N', 'C(=O)(O)O']
>>>
Use “find” to find the
start of a substring.
Start looking at position 10.
Find returns -1 if it couldn’t
find a match.
Split the string into parts
with “.” as the delimiter
27. String operators: in, not in
if "Br" in “Brother”:
print "contains brother“
email_address = “clin”
if "@" not in email_address:
email_address += "@brandeis.edu“
28. String Method: “strip”, “rstrip”, “lstrip” are ways to
remove whitespace or selected characters
>>> line = " # This is a comment line n"
>>> line.strip()
'# This is a comment line'
>>> line.rstrip()
' # This is a comment line'
>>> line.rstrip("n")
' # This is a comment line '
>>>
30. Control flow: if
x = int(raw_input("Please enter #:"))
if x < 0:
x = 0
print 'Negative changed to zero'
elif x == 0:
print 'Zero'
elif x == 1:
print 'Single'
else:
print 'More'
» no case statement
31. Control flow: for
a = ['cat', 'window', 'defenestrate']
for x in a:
print x, len(x)
»no arithmetic progression, but
range(10) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in range(len(a)):
print i, a[i]
»do not modify the sequence being iterated over
32. Loop Control Statements
break Jumps out of the closest
enclosing loop
continue Jumps to the top of the closest
enclosing loop
pass Does nothing, empty statement
placeholder
33. Loops: break, continue, else
»break and continue like C
»else after loop exhaustion
for n in range(2,10):
for x in range(2,n):
if n % x == 0:
print n, 'equals', x, '*', n/x
break
else:
# loop fell through without finding a factor
print n, 'is prime'
34. Loop example
» import fnmatch
» import os
» images = ['*.jpg', '*.jpeg', '*.png', '*.tif', '*.tiff']
» matches = []
» for root, dirnames, filenames in os.walk('C:'):
» for extensions in images:
» for filename in fnmatch.filter(filenames, extensions):
» print filename
» matches.append(os.path.join(root, filename))
Simple Matching
fnmatch() compares a single file name against a pattern and returns a
Boolean indicating whether or not they match.
Filtering
To test a
sequence of
filenames, you
can use filter().
It returns a list
of the names
that match the
pattern
argument.
35. Loop example
» # Python program to check if the input number is prime or not
» num = 407
» # take input from the user
» # num = int(input("Enter a number: "))
» # prime numbers are greater than 1
» if num > 1:
» # check for factors
» for i in range(2,num):
» if (num % i) == 0:
» print(num,"is not a prime number")
» print(i,"times",num//i,"is",num)
» break
» else:
» print(num,"is a prime number")
»
» # if input number is less than
» # or equal to 1, it is not prime
» else:
» print(num,"is not a prime number")
To understand this example, you should have
the knowledge of following Python
programming topics:
• Python if...else Statement
• Python for Loop
• Python break and continue
A positive integer greater than 1 which has no other factors
except 1 and the number itself is called a prime number. 2,
3, 5, 7 etc. are prime numbers as they do not have any other
factors. But 6 is not prime (it is composite) since, 2 x 3 = 6.
37. List comprehensions
»Create lists without map(), filter(), lambda
»= expression followed by for clause + zero or more for or of clauses
>>> vec = [2,4,6]
>>> [3*x for x in vec]
[6, 12, 18]
>>> [{x: x**2} for x in vec}
[{2: 4}, {4: 16}, {6: 36}]
38. List comprehensions
»cross products:
>>> vec1 = [2,4,6]
>>> vec2 = [4,3,-9]
>>> [x*y for x in vec1 for y in vec2]
[8,6,-18, 16,12,-36, 24,18,-54]
>>> [x+y for x in vec1 and y in vec2]
[6,5,-7,8,7,-5,10,9,-3]
>>> [vec1[i]*vec2[i] for i in range(len(vec1))]
[8,12,-54]
40. Tuples: sort of an immutable list
>>> yellow = (255, 255, 0) # r, g, b
>>> one = (1,)
>>> yellow[0]
>>> yellow[1:]
(255, 0)
>>> yellow[0] = 0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
Very common in string interpolation:
>>> "%s lives in %s at latitude %.1f" % ("Andrew", "Sweden", 57.7056)
'Andrew lives in Sweden at latitude 57.7'
41. Tuples and sequences
»lists, strings, tuples: examples of sequence type
»tuple = values separated by commas
>>> t = 123, 543, 'bar'
>>> t[0]
123
>>> t
(123, 543, 'bar')
42. Tuples
»Tuples may be nested
>>> u = t, (1,2)
>>> u
((123, 542, 'bar'), (1,2))
»like strings, immutable can't assign to individual items
44. Tuples
»sequence unpacking distribute elements across variables
>>> t = 123, 543, 'bar'
>>> x, y, z = t
>>> x
123
»packing always creates tuple
»unpacking works for any sequence
45. Dictionaries
»associative arrays
»indexed by keys
»keys are any immutable type: e.g., tuples
»but not lists (mutable!)
»uses 'key: value' notation
>>> tel = {'hgs' : 7042, 'lennox': 7018}
>>> tel['cs'] = 7000
>>> tel
46. Dictionaries
»no particular order
»delete elements with del
>>> del tel['foo']
»keys() method unsorted list of keys
>>> tel.keys()
['cs', 'lennox', 'hgs']
»use has_key() to check for existence
>>> tel.has_key('foo')
0
47. Dictionaries
» prices = {'apple': 0.40, 'banana': 0.50}
» my_purchase = {
» 'apple': 1,
» 'banana': 6}
» grocery_bill = sum(prices[fruit] *
my_purchase[fruit]
» for fruit in my_purchase)
» print 'I owe the grocer $%.2f' % grocery_bill
48. Defining functions
def fib(n):
"""Print a Fibonacci series up to n."""
a, b = 0, 1
while b < n:
print b,
a, b = b, a+b
>>> fib(2000)
» First line is docstring
» first look for variables in local, then global
» need global to assign global variables
50. Importing and Modules
» Use classes & functions defined in another file
» A Python module is a file with the same name
(plus the .py extension)
» Like Java import, C++ include
» Three formats of the command:
import somefile
from somefile import *
from somefile import className
» The difference? What gets imported from the file
and what name refers to it after importing
51. import …
import somefile
» Everything in somefile.py gets imported.
» To refer to something in the file, append the text
“somefile.” to the front of its name:
somefile.className.method(“abc”)
somefile.myFunction(34)
52. from … import *
from somefile import *
» Everything in somefile.py gets imported
» To refer to anything in the module, just use its
name. Everything in the module is now in the
current namespace.
» Take care! Using this import command can easily
overwrite the definition of an existing function or
variable!
className.method(“abc”)
myFunction(34)
53. from … import …
from somefile import className
» Only the item className in somefile.py gets
imported.
» After importing className, you can just use it
without a module prefix. It’s brought into the
current namespace.
» Take care! Overwrites the definition of this name
if already defined in the current namespace!
className.method(“abc”) imported
myFunction(34) Not
imported
56. Target
»Web Scraping
The need and importance of extracting data
from the web is becoming increasingly loud and
clear.
There are several ways to extract information
from the web.
Use of APIs being probably the best way to
extract data from a website. If you can get what
you need through an API, it is almost always
preferred approach over web scrapping.
57. Target
»Web Scraping
Sadly, not all websites provide an API.
Some do it because they do not want
the readers to extract huge information
in structured way, while others don’t
provide APIs due to lack of technical
knowledge. What do you do in these
cases?
Well, we need to scrape the website to
fetch the information.
58. Target
»Ok, but what is Web Scraping?
Web scraping is a computer software technique of extracting information from
websites. This technique mostly focuses on the transformation of unstructured
data (HTML format) on the web into structured data (database or spreadsheet).
You can perform web scrapping in various ways…
We’ll resort to Python because of its ease and rich ecosystem. It has a library
known as ‘BeautifulSoup’ which assists this task.
In this lesson, I’ll show you the easiest way to learn web scraping using python
programming.
59. Downloading Files from the Web with the requests Module
» The requests module lets you easily download files from the Web without
having to worry about complicated issues such as network errors,
connection problems, and data compression.
» The requests module doesn’t come with Python, so you’ll have to install it
first. From the command line, run pip install requests.
» Next, do a simple test to make sure the requests module installed itself
correctly. Enter the following into the interactive shell:
>>> import requests
» If no error messages show up, then the requests module has been
successfully installed.
60. Downloading a Web Page with the requests.get() Function
»The requests.get() function takes a string of a URL to download.
»By calling type() on requests.get()’s return value, you can see that it
returns a Response object, which contains the response that the web
server gave for your request…
61. Downloading a Web Page with the requests.get() Function
The Complete Works of William Shakespeare
The Tragedy of Romeo and Juliet
The Library of the Future Complete Works of William Shakespeare
Library of the Future is a TradeMark (TM) of World Library Inc.
<<THIS ELECTRONIC VERSION OF THE COMPLETE WORKS OF WILLIAM
SHAKESPEARE IS COPYRIGHT 1990-1993 BY WORLD LIBRARY, INC., AND IS
PROVIDED BY PROJECT GUTENBERG ETEXT OF CARNEGIE MELLON UNIVERSITY
WITH PERMISSION. ELECTRONIC AND MACHINE READABLE COPIES MAY BE
DISTRIBUTED SO LONG AS SUCH COPIES (1) ARE FOR YOUR OR OTHERS
PERSONAL USE ONLY, AND (2) ARE NOT DISTRIBUTED OR USED
COMMERCIALLY. PROHIBITED COMMERCIAL DISTRIBUTION INCLUDES BY ANY
SERVICE THAT CHARGES FOR DOWNLOAD TIME OR FOR MEMBERSHIP.>>
1595
THE TRAGEDY OF ROMEO AND JULIET
by William Shakespeare
62. Libraries required for web scraping
» Urllib2: It is a Python module which can be used for
fetching URLs.
It defines functions and classes to help with URL actions
(basic and digest authentication, redirections, cookies,
etc).
For more detail refer to the documentation page.
» BeautifulSoup: It is an incredible tool for pulling out
information from a webpage.
You can use it to extract tables, lists, paragraph and you
can also put filters to extract information from web
pages.
You can look at the installation instruction in
its documentation page.
63. Basics – Get familiar with HTML (Tags)
<!DOCTYPE html> : HTML documents must start with a type
declaration
HTML document is contained between <html> and </html>
The visible part of the HTML document is
between <body> and </body>
HTML headings are defined with the <h1> to <h6> tags
HTML paragraphs are defined with the <p> tag
65. Basics – Get familiar with HTML (Tags)
»Other useful HTML tags are:
HTML links are defined with
the <a> tag,
“<a href=“http://www.test.com”
>This is a link for test.com</a>”
HTML tables are defined
with<Table>, row as <tr> and
rows are divided into data as
<td>
66. Scrapping a web Page using BeautifulSoup
»Here, I am scraping data
from a Wikipedia page.
Our final goal is to extract list
of state, union territory
capitals in India. And some
basic detail like
establishment, former capital
and others form
this wikipedia page.
Let’s learn with doing this
project step wise step…
68. It’s all objects…
»Everything in Python is really an object.
We’ve seen hints of this already…
“hello”.upper()
list3.append(‘a’)
dict2.keys()
These look like Java or C++ method calls.
New object classes can easily be defined in addition to these built-
in data-types.
»In fact, programming in Python is typically done in an object oriented
fashion.
69. Defining a Class
»A class is a special data type which defines how to
build a certain kind of object.
»The class also stores some data items that are
shared by all the instances of this class
»Instances are objects that are created which follow
the definition given inside of the class
»Python doesn’t use separate class interface
definitions as in some languages
»You just define the class and then use it
70. Methods in Classes
»Define a method in a class by including function
definitions within the scope of the class block
»There must be a special first argument self in all
of method definitions which gets bound to the
calling instance
»There is usually a special method called
__init__ in most classes
»We’ll talk about both later…
71. A simple class def: student
class student:
“““A class representing a
student ”””
def __init__(self,n,a):
self.full_name = n
self.age = a
def get_age(self):
return self.age
72. Instantiating Objects
» There is no “new” keyword as in Java.
» Just use the class name with ( ) notation and assign the result to a variable
» __init__ serves as a constructor for the class. Usually does some
initialization work
» The arguments passed to the class name are given to its __init__()
method
» So, the __init__ method for student is passed “Bob” and 21 and the new
class instance is bound to b:
b = student(“Bob”, 21)
73. Constructor: __init__
»An __init__ method can take any number of arguments.
»Like other functions or methods, the arguments can be defined with
default values, making them optional to the caller.
»However, the first argument self in the definition of __init__ is
special…
74. Self
»The first argument of every method is a reference
to the current instance of the class
»By convention, we name this argument self
»In __init__, self refers to the object currently
being created; so, in other class methods, it refers
to the instance whose method was called
»Similar to the keyword this in Java or C++
»But Python uses self more often than Java uses
this
75. Self
»Although you must specify self explicitly when defining the
method, you don’t include it when calling the method.
»Python passes it for you automatically
Defining a method: Calling a method:
(this code inside a class definition.)
def set_age(self, num): >>> x.set_age(23)
self.age = num
76. Deleting instances: No Need to “free”
»When you are done with an object, you don’t have
to delete or free it explicitly.
»Python has automatic garbage collection.
»Python will automatically detect when all of the
references to a piece of memory have gone out of
scope. Automatically frees that memory.
»Generally works well, few memory leaks
»There’s also no “destructor” method for classes
77. Definition of student
class student:
“““A class representing a student
”””
def __init__(self,n,a):
self.full_name = n
self.age = a
def get_age(self):
return self.age
78. Traditional Syntax for Access
>>> f = student(“Bob Smith”, 23)
>>> f.full_name # Access attribute
“Bob Smith”
>>> f.get_age() # Access a method
23
79. Two Kinds of Attributes
» The non-method data stored by objects are called attributes
» Data attributes
Variable owned by a particular instance of a class
Each instance has its own value for it
These are the most common kind of attribute
» Class attributes
Owned by the class as a whole
All class instances share the same value for it
Called “static” variables in some languages
Good for (1) class-wide constants and (2) building
counter of how many instances of the class have been
made
80. Data Attributes
»Data attributes are created and initialized by an __init__()
method.
Simply assigning to a name creates the attribute
Inside the class, refer to data attributes using self
for example, self.full_name
class teacher:
“A class representing teachers.”
def __init__(self,n):
self.full_name = n
def print_name(self):
print self.full_name
81. Class Attributes
» Because all instances of a class share one copy of a class
attribute, when any instance changes it, the value is
changed for all instances
» Class attributes are defined within a class definition and
outside of any method
» Since there is one of these attributes per class and not one
per instance, they’re accessed via a different notation:
Access class attributes using self.__class__.name notation --
This is just one way to do this & the safest in general.
class sample: >>> a = sample()
x = 23 >>> a.increment()
def increment(self): >>> a.__class__.x
self.__class__.x += 1 24
82. Data vs. Class Attributes
class counter:
overall_total = 0
# class attribute
def __init__(self):
self.my_total = 0
# data attribute
def increment(self):
counter.overall_total =
counter.overall_total + 1
self.my_total =
self.my_total + 1
>>> a = counter()
>>> b = counter()
>>> a.increment()
>>> b.increment()
>>> b.increment()
>>> a.my_total
1
>>> a.__class__.overall_total
3
>>> b.my_total
2
>>> b.__class__.overall_total
3
83. Subclasses
»A class can extend the definition of another class
Allows use (or extension ) of methods and attributes already defined in the
previous one.
New class: subclass. Original: parent, ancestor or superclass
»To define a subclass, put the name of the superclass in parentheses
after the subclass’s name on the first line of the definition.
Class Cs_student(student):
Python has no ‘extends’ keyword like Java.
Multiple inheritance is supported.
84. Redefining Methods
»To redefine a method of the parent class, include a new definition
using the same name in the subclass.
The old code won’t get executed.
»To execute the method in the parent class in addition to new code for
some method, explicitly call the parent’s version of the method.
parentClass.methodName(self, a, b, c)
The only time you ever explicitly pass ‘self’ as an argument is when calling a
method of an ancestor.
85. Definition of a class extending student
Class Student:
“A class representing a student.”
def __init__(self,n,a):
self.full_name = n
self.age = a
def get_age(self):
return self.age
Class Cs_student (student):
“A class extending student.”
def __init__(self,n,a,s):
student.__init__(self,n,a) #Call __init__ for student
self.section_num = s
def get_age(): #Redefines get_age method entirely
print “Age: ” + str(self.age)