4. Лучшее 2015
«Афиша-Рестораны» и «Рамблер.Почта» вошли в список
25 лучших приложений 2015 года для iOS по версии
редакции AppStore, официального онлайн-магазина Apple.
У «Афиши-Рестораны» — 15-е место в рейтинге,
«Рамблер.Почта» — на 24-м.
http://lenta.ru/news/2015/12/09/rambler/
(http://lenta.ru/news/2015/12/09/rambler/)
5. Контакты
В группе компаний Rambler&Co всегда есть
открытые вакансии для тех, кто хочет
профессионально расти и развиваться, занимаясь
тем, что по-настоящему нравится
hr@rambler-co.ru
&www.rambler-co.ru/jobs (www.rambler-co.ru/jobs)
7. IPython
2001
Все в консоли, отображение графиков в новом окне и т.д.
IPython Notebook pre 1.0
18 декабря 2011
8. 8 августа 2013 - IPython 1.0
1 апреля 2014 - IPython 2.0
Появились виджеты
27 февраля 2015 - IPython 3.0
- 54
IPython kernels for other languages (https://github.com/ipython/ipython/wiki/IPython-
kernels-for-other-languages)
18. Немного магии
In [14]: %lsmagic
Out[14]: Available line magics:
%alias %alias_magic %autocall %automagic %autosave %bookmark %cat %cd %clear %co
Available cell magics:
%%! %%HTML %%SVG %%bash %%capture %%debug %%file %%html %%javascript %%latex %%
Automagic is ON, % prefix IS NOT needed for line magics.
In [15]: %quickref
19. pwd
Current working directory path
In [16]: %pwd
Out[16]: '/Users/l.tonkikh/ipython'
What in inside?
In [17]: import os
os.getcwd(), os.path.realpath('.')
# or smth else?
Out[17]: ('/Users/l.tonkikh/ipython', '/Users/l.tonkikh/ipython')
21. In [19]: from IPython.core.magics.osm import OSMagics
magic = OSMagics()
magic.pwd()
Out[19]: '/Users/l.tonkikh/ipython'
In [20]: %psource magic.pwd
In [21]: %psource %psource
In [22]: %psource %pwd
28. One more example
In [38]: ! pip freeze | grep ipython
ipython==4.0.3
ipython-genutils==0.1.0
In [39]: !! pip freeze | grep ipython
Out[39]: ['ipython==4.0.3', 'ipython-genutils==0.1.0']
31. In [41]: %who
OSMagics a env files magic os q
In [42]: %who_ls
Out[42]: ['OSMagics', 'a', 'env', 'files', 'magic', 'os', 'q']
32. In [43]: %who_ls dict
Out[43]: ['env']
In [44]: %whos
Variable Type Data/Info
-------------------------------------
OSMagics MetaHasTraits <class 'IPython.core.magics.osm.OSMagics'>
a SList ['You are using pip versi<...>ipython-genutils==0.1.0']
env dict n=39
files SList ['Untitled.ipynb', 'docs'<...>', 'test.png', 'test.py']
magic OSMagics <IPython.core.magics.osm.<...>cs object at 0x106353160>
os module <module 'os' from '/Libra<...>3.5/lib/python3.5/os.py'>
q bool True
33. psearch
Поиск по имени переменной
In [63]: a1 = 1
a2 = 'a2'
In [64]: %psearch a*
In [66]: %psearch -e builtin a*
# a1
# a2
In [67]: %psearch -e builtin a* int
34. Logging
In [69]: %logstate
Logging has not been activated.
In [70]: %logstart
Activating auto-logging. Current session state plus future input saved.
Filename : ipython_log.py
Mode : rotate
Output logging : False
Raw input log : False
Timestamping : False
State : active
39. In [71]: %%python3
import sys
print(sys.version)
3.5.0 (v3.5.0:374f501f4567, Sep 12 2015, 11:00:19)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
In [6]: %%perl
print(sqrt(4))
2
In [3]: %%ruby
include Math
puts sqrt(4)
2.0
40. Write & Sharing
In [254]: %%writefile -a overwritten.py
a = 'Hi'
# You are in Good Company
Writing overwritten.py
In [255]: %pastebin -d 'test file' overwritten.py
Out[255]: 'https://gist.github.com/baaa3be4614b1c26d6f1'
41. Time
Время выполнения
In [279]: %time x = sum(range(10000))
CPU times: user 365 µs, sys: 2 µs, total: 367 µs
Wall time: 370 µs
In [278]: %timeit x = sum(range(10000))
1000 loops, best of 3: 208 µs per loop
In [42]: %%timeit -n 1000
x = range(10000)
max(x)
1000 loops, best of 3: 320 µs per loop
48. Github Commits
Introducing the New GitHub Graphs - April 25, 2012 link (https://github.com/blog/1093-
introducing-the-new-github-graphs)
49. In [4]: from github import Github
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
%matplotlib inline
In [3]: GITHUB_USER = os.getenv('GITHUB_USER')
GITHUB_TOKEN = os.getenv('GITHUB_TOKEN')
REPO_PATH = 'python/cpython'
In [8]: g = Github(GITHUB_USER, GITHUB_TOKEN)
repo = g.get_repo(REPO_PATH)
df = pd.DataFrame({'date': pd.Series((i.commit.committer.date for i in repo.get_commits()
stats = df.groupby(pd.Grouper(key='date', freq='M')).size()
50. In [35]: plt.figure(figsize=(16,4))
stats.plot(title='Stats of commits', label='Commits', legend=True)
plt.xlabel('Date'); plt.ylabel('Commits')
plt.suptitle(REPO_PATH, y=1.05, fontsize=14, fontweight='bold')
Out[35]: <matplotlib.text.Text at 0x1789e32b0>
51. ipyparallel
Powerful architecture for parallel and distributed computing
Single program, multiple data (SPMD) parallelism.
Multiple program, multiple data (MPMD) parallelism.
Message passing using MPI.
Task farming.
Data parallel.
Combinations of these approaches.
Custom user de ned approaches
56. Controller
provide an interface for working with a set of engines (Direct or LoadBalanced)
collection of processes to which IPython engines and clients can connect
Controller = Hub + Schedulers
57. Hub
Center of a Cluster
Process that keeps track of engine connections, schedulers, clients
58. Schedulers
All actions that can be performed on the engine go through a Scheduler
provide a fully asynchronous interface to a set of engines
59. In [ ]: ! ipcluster start -n 4
In [3]: %%bash
python not_track/habraparse.py save_favs_list stleon not_track/test.txt
python not_track/habraparse.py save_favs_list --gt stleon not_track/test1.txt
python not_track/habraparse.py save_favs_list --mm stleon not_track/test2.txt
cat not_track/test1.txt not_track/test2.txt >> not_track/test.txt
In [47]: from ipyparallel import Client, require
client = Client()
dview = client[:]
In [48]: dview
Out[48]: <DirectView [0, 1, 2, 3]>
In [49]: links = (link.strip() for link in open('not_track/test.txt'))
In [50]: dview.scatter('links', list(links))
Out[50]: <AsyncResult: finished>
60. In [51]: len(dview['links'])
Out[51]: 4
In [52]: @dview.remote(block=False)
@require('requests', 'bs4')
def tag_maker():
tags = {}
for link in links:
soup = bs4.BeautifulSoup(requests.get(link).text, 'html.parser')
for i in soup.findAll("a", rel="tag"):
tag = i.string.lower()
tags[tag] = 1 + tags.get(tag, 0)
return tags
In [53]: tags = tag_maker().result
In [1]: # tags
61. In [55]: from collections import Counter
from functools import reduce
from operator import add
new_tags = dict(reduce(add, (Counter(tag) for tag in tags)))
In [56]: len(new_tags.keys())
Out[56]: 827
In [ ]: from wordcloud import WordCloud
wordcloud = WordCloud(width=1920, height=1080, scale=1,
font_path='/Library/Fonts/Verdana.ttf',
max_words=len(new_tags.keys()))
wordcloud.generate_from_frequencies(new_tags.items())