Download as PDF, PPTX
Uploaded byRyuji Tamagawa
20170210 sapporotechbar7
1. PyData is a community for users and developers of open-source data tools in Python including NumPy, Pandas, SciPy, scikit-learn, IPython, and Jupyter. 2. Pandas is a software library written for data manipulation and analysis in Python, built on top of NumPy and SciPy. It provides data structures and operations for working with relational or labeled data and time series. 3. Jupyter Notebook is an open-source web application that allows users to create and share documents that contain live code, equations, visualizations and explanatory text. It supports over 40 programming languages including Python, R and Julia.
More Related Content
![[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社ＨＰＣソリューションズ ...](https://cdn.slidesharecdn.com/ss_thumbnails/dbtechshowcase-sapporo-2017-iisaka-170707075724-thumbnail.jpg?width=640&height=640&fit=bounds)
20170210 sapporotechbar7
- 1. PyData & Apache Spark 2017/ 2 / 10 Sapporo TechBar #7 @
- 2. ▸ facebook :Ryuji Tamagawa ▸ Twitter : tamagawa_ryuji ▸ FB techbar ▸ FB ▸ Twitter
- 4.
- 6.
- 7.
- 8. 1 / 5: PyData
- 9. 1 / 5: PyData PyData.org
- 10. 1 / 5: PyData PyData Anaconda Python Blaze NumPy and pandas interface to Big Data'. dask Bokeh Canopy Python IPython matplotlib PyData nose numba JIT NumPy PyData Scipy PyData Statsmodels SymPy pandas NumPy SciPy scikit-image scikit-learn PyData
- 11.
- 12. 2 / 5: pandas pandas ▸ NumPy SciPy ▸ DataFrame ▸
- 13. 2 / 5: pandas pandas Wes McKinney
- 14. 2 / 5: pandas DataFrame
- 15. 2 / 5: pandas
- 16. 2 / 5: pandas ▸ Python ▸ ▸ PyData pandas
- 17.
- 18. 3 /5 :Jupyter Notebook IPython Notebook ▸ Jupyter Notebook ▸ Julia Python R ▸ JupyterCon
- 19. 3 /5 :Jupyter Notebook
- 20. 3 /5 :Jupyter Notebook
- 21. 3 /5 :Jupyter Notebook pandas / matplotlib
- 22. 3 /5 :Jupyter Notebook Interactive Widget
- 23. 3 /5 :Jupyter Notebook ▸ Learning Jupyter
- 24.
- 25. 4 / 5: Apache Spark Hadoop ▸ MapReduce Spark ▸ 2010 Hadoop = MapReduce + HDFS ▸ Hadoop OS HDFS Hive e.t.c. HBaseMapReduce YARN Impala e.t.c in- memory SQL engine Spark Spark Streaming, MLlib, GraphX, Spark SQL) Hadoop HDFS S3 YARN Mesos /
- 26. 4 / 5: Apache Spark Apache Spark PyData pandas Apache Spark pandas JVM Python × dask I/O Scala Java Python R JVM Python
- 27. 4 / 5: Apache Spark Spark ▸ ▸ ▸ 1 PC Hadoop / MapReduce
- 28. 4 / 5: Apache Spark DataFrame
- 29. 4 / 5: Apache Spark ▸ ▸ SSD ▸ Spark Parquet ▸ Performance comparison of different file formats and storage engines in the Hadoop ecosystem ▸ Parquet Python
- 30. 4 / 5: Apache Spark Apache Spark ▸ ▸ Parquet ▸ ▸
- 31.
- 32. Machine Learning ▸ ▸ scikit-learn ▸Spark MLlib / ML ▸ ▸ TensorFlow ▸ Python
- 33. 2017 and thefuture
- 34. 5/5 : 2017and the future PyData ▸ ▸ Spark - pandas ▸ pandas → Spark …
- 35. 5/5 : 2017and the future Wes blog ▸ pandas Apache Arrow ▸ Blog ▸ PyData Blog Wes OK ▸ 2017 : pandas, Arrow, Feather, Parquet, Spark, Ibis http://qiita.com/tamagawa-ryuji/items/deb3f63ed4c7c8065e81
- 36. 5/5 : 2017and the future High speed Apache Parquet for Python ▸ Parquet ▸ Spark ▸ Python ▸ Fastparquet ▸ pyarrow
- 37. 5/5 : 2017and the future : apache arrow ▸ apache arrow ▸ PyData / OSS ▸ /