Data wrangling week2

•

0 likes•186 views

The document discusses a lesson on data wrangling in Python. It covers basics of Python like its history, versions, and features. It also discusses data types in Python like numeric, string, and sequence types. Specifically, it covers integers, floats, strings, lists and operations on sequences like strings and lists. The document provides instructions on installing Miniconda and Jupyter Lab as well as examples of Python code for hello world, variables, and operations. The next week's lesson will cover dictionaries and control statements.

Data Wrangling
Week 2
Dr. Ferdin Joe John Joseph
Faculty of Information Technology
Thai – Nichi Institute of Technology, Bangkok

Today’s Lesson
• Basics of Python
• Data Types
• Variables
• Operators and Operands
• Lists
• Demonstration
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Basics of Python
• Python is available in version 2 and 3
• Version 3 is highly recommended
• Version 2 is used to run some old frameworks
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Basics of Python
• Development started in the 1980’s by Guido van Rossum.
• Only became popular in the last decade or so.
• Interpreted, very-high-level programming language.
• Supports a multitude of programming paradigms.
• OOP, functional, procedural, logic, structured, etc.
• General purpose.
• Very comprehensive standard library includes numeric modules,
crypto services, OS interfaces, networking modules, GUI support,
development tools, etc.
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Features of Python
• Easy to learn.
• Supports quick development.
• Cross-platform.
• Open Source.
• Extensible.
• Embeddable.
• Large standard library and active community.
• Useful for a wide variety of applications.
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Getting Started
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Recommended way
• Use Miniconda
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Miniconda Installation
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Jupyter Lab Installation
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Jupyter Lab Installation
• Enter the following in the command prompt
conda install –c conda-forge jupyterlab
Respond ‘y’ when prompted
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Jupyter Lab – Getting Started
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Jupyter Lab
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Hello World
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Python - Datatype
• All datatypes are supported
• No need to mention integer of string
• Flexible in defining variable
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Python - Datatype
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Python – Operations and operands
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Python – Operators and Operands
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Variables: Numeric Type
• The subtypes are int, long, float and complex.
• Mixed arithmetic is supported, with the “narrower” type widened to
that of the other.
• The same rule is used for mixed comparisons.
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Variables: Numeric Type
• int: equivalent to Java’s long int in 2.x but unlimited in 3.x.
• float: equivalent to Java’s doubles.
• long: unlimited in 2.x and unavailable in 3.x.
• Complex: Complex numbers
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Numeric Type
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Variables: Sequence Type
• There are seven sequence subtypes: strings, Unicode strings, lists,
tuples, bytearrays, buffers, and xrange objects.
• The most commonly used sequence data types are strings, lists, and
tuples. The xrange data type finds common use in the construction of
enumeration-controlled loops. The others are used less commonly.
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Sequence Type: Strings
• Created by simply enclosing characters in either single- or double-
quotes.
• It’s enough to simply assign the string to a variable. Strings are
immutable.
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Strings
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Lists
• Arrays in Python
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Lists
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Sequence operations
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Exercise
• Practice all the codes in Google Colab
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

Lesson for Next Week
• Dictionaries
• Control Statements
• Demonstration
Faculty of Information Technology, Thai - Nichi Institute of
Technology, Bangkok

This document outlines a lesson on data wrangling using Pandas. It introduces Pandas as a tool for exploring, cleaning, and transforming data. Pandas allows users to read data from files like CSV and Excel, calculate statistics, clean data by removing missing values, and store cleaned data back into files. The lesson demonstrates how to install Pandas, read data from various sources into a DataFrame, explore the data, handle missing values, and more. The goal is to introduce the main capabilities and functions of the Pandas library.

Data wrangling week 10

Ferdin Joe John Joseph PhD

This document provides an overview of outlier detection techniques. It begins by defining outliers as data points that are considerably different from the majority of the data set. Several approaches to outlier detection are then discussed, including statistical-based methods using distributions and depth, deviation-based methods like sequential exceptions and OLAP data cubes, and distance-based methods using indexes and nearest neighbors. Specific algorithms are explained for each approach.

Data wrangling week3

Ferdin Joe John Joseph PhD

This document outlines a lesson plan on data wrangling that covers dictionaries, control statements like while loops and if/else statements, and user-defined functions in Python. It includes examples and exercises for working with dictionaries to map keys to values, using control statements with dictionaries, defining functions, and taking user input. The lesson is intended for an information technology class at Thai-Nichi Institute of Technology in Bangkok and instructs students to complete exercises on generating Fibonacci sequences and calculating factorials using Google Collab.

Data wrangling week 6

Ferdin Joe John Joseph PhD

The document outlines a lesson on data wrangling that includes an introduction to XML, parsing XML using Python, and parsing text files using Python. It discusses using XML for storing and transmitting data and covers XML theories. The lesson demonstrates parsing an XML file into a Pandas dataframe and performing queries and analysis on the XML data. It also discusses parsing text files like CSV into a dataframe and performing similar analysis.

Data wrangling week1

Ferdin Joe John Joseph PhD

This document outlines the course contents for a data wrangling class. It introduces data wrangling and cleaning as the process of converting raw data into a usable format. The course will cover Python programming, Jupyter notebooks, common Python libraries like Pandas and NumPy, and installing packages. Students will learn the steps of data wrangling including importing, merging, standardizing, and exporting data. The instructor will demonstrate working with data in Jupyter notebooks.

Week 9: Programming for Data Analysis

Ferdin Joe John Joseph PhD

This document discusses different types of linear regression models including simple, multiple, and polynomial linear regression. It provides code examples for implementing linear regression using scikit-learn and statsmodels. Key steps covered include importing packages, providing data, creating and fitting regression models, obtaining results like coefficients and metrics, making predictions on new data, and visualizing models. Types of linear regression covered are simple linear regression with one variable, multiple linear regression with two or more variables, and polynomial regression with higher degree polynomials.

Blockchain Technology - Week 6 - Role of Cryptography in Blockchain

Ferdin Joe John Joseph PhD

Week 1: Programming for Data Analysis

Ferdin Joe John Joseph PhD

This document outlines the topics to be covered in a programming for data analysis course at the Thai-Nichi Institute of Technology in Bangkok. The course will introduce Pandas, a data analysis library in Python. It will cover reading data from Excel/CSV files, calculating statistics, cleaning data, and visualizing data with Matplotlib. Specific topics will include reading and writing data, working with indexes, querying data, handling missing values, and correlating variables. Students will learn to import data, explore datasets, clean and transform data, and store results back into files.

This document summarizes a lesson on data analysis techniques in pandas including pivoting, binning, and replacing/renaming data. It discusses pivoting tables in pandas, binning continuous numeric data into buckets for analysis using cut and qcut, and naming the bins. Methods of equal frequency and equal width binning are also covered. The lesson concludes with an example of creating a pivot table and visualizing binning of data using cut, qcut, and named bins.

Blockchain Technology - Week 5 - Cryptography and Steganography

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 2 - Blockchain Terminologies

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 3 - FinTech and Cryptocurrencies

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 4 - Hyperledger and Smart Contracts

Ferdin Joe John Joseph PhD

This document outlines a class on blockchain technology and smart contracts. It discusses hyperledgers, their requirements and architecture. Hyperledger is introduced as an open source distributed ledger framework. Smart contracts are defined and their properties, features, interpretation using Ricardian contracts, and applications are explained. Various smart contract languages are also mentioned.

Blockchain Technology - Week 1 - Introduction to Blockchain

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 10 - CAP Teorem, Byzantines General Problem

Ferdin Joe John Joseph PhD

Programming for Data Analysis: Week 4

Ferdin Joe John Joseph PhD

This document discusses permutations, combinations, and binomial coefficients. It provides examples of calculating permutations and combinations, as well as properties like Pascal's identity. It also discusses combinations with repetition and solving combinatorics problems. The document is a lesson plan on programming for data analysis that covers permutations, combinations, and a laboratory session.

Week 8: Programming for Data Analysis

Ferdin Joe John Joseph PhD

This document discusses feature engineering techniques for data analysis. It covers feature selection, construction, and engineering. Specific techniques discussed include feature imputation, handling outliers, binning, log transforms, one-hot encoding, grouping, splitting, scaling, and extracting date features. The document provides examples and explanations of these techniques to transform raw data into more useful features for machine learning models.

Week 10: Programming for Data Analysis

Ferdin Joe John Joseph PhD

This document summarizes a lesson on logistic regression for data analysis. It discusses logistic regression as a machine learning classification algorithm used to predict categorical dependent variables. It then describes assumptions of logistic regression, provides examples of its practical use, and outlines the steps taken in building a logistic regression model for a direct marketing dataset, including data preprocessing, model training and evaluation.

Python Tutorial For Beginners | Python Crash Course - Python Programming Lang...

Edureka!

** Python Certification Training: https://www.edureka.co/python ** This Edureka tutorial on "Python Tutorial for Beginners" (Python Blog Series: https://goo.gl/nKQJHQ) covers all the basics of Python. It includes python programming examples, so try it yourself and mention in the comments section if you have any doubts. Following are the topics included in this PPT: Introduction to Python Reasons to choose Python Installing and running Python Development Environments Basics of Python Programming Starting with code Python Operators Python Lists Python Tuples Python Sets Python Dictionaries Conditional Statements Looping in Python Python Functions Python Arrays Classes and Objects (OOP) Conclusion Follow us to never miss an update in the future. Instagram: https://www.instagram.com/edureka_learning/ Facebook: https://www.facebook.com/edurekaIN/ Twitter: https://twitter.com/edurekain LinkedIn: https://www.linkedin.com/company/edureka

MongoDB and BlockChain

Massimo Brignoli

Blockchains promise significant benefits but also face challenges to widespread adoption. Some key challenges are a lack of interoperability between different blockchain networks and platforms, as well as issues around integration with existing systems and scaling to meet enterprise needs. Interledger aims to address the interoperability challenge by providing a protocol for connecting various ledger systems, including blockchains, to enable asset transfers between them. This allows for a more connected "Internet of Value" and new business opportunities across platforms.

Blockchain .pptx

ManishKumar7730

Python vs rust

Simplilearn

Web3.0の話をしよう！ (NFT編).pdf

AtsushiSasaki9

ブロックチェーンを学ぶ　公開版

YamashitaKatsushi

Text mining

Pankaj Thakur

The document provides an overview of text mining, including: 1. Text mining analyzes unstructured text data through techniques like information extraction, text categorization, clustering, and summarization. 2. It differs from regular data mining as it works with natural language text rather than structured databases. 3. Text mining has various applications including security, biomedicine, software, media, business and more. It faces challenges in representing meaning and context from unstructured text.

Introdução às criptomoedas: criando a sua própria moeda como o Bitcoin!

Alex Camargo

C MCQ

amutharajan10

The document contains questions related to C programming language. It tests knowledge on topics like: - The creator of C language and where it was developed - How C programs are converted to machine language - C language keywords, variables, data types, operators etc. - Control structures like if-else, for loop, while loop, switch case - Input/output functions like printf, scanf - Arrays, strings and other basic concepts The questions have multiple choice answers to test understanding of fundamental C programming concepts.

Best Python Libraries For Data Science & Machine Learning | Edureka

Edureka!

This document provides an overview of popular Python libraries for data science and machine learning tasks. It discusses libraries for statistical analysis (NumPy, SciPy, Pandas, StatsModels), data visualization (Matplotlib, Seaborn, Plotly, Bokeh), machine learning (Scikit-learn, XGBoost, Eli5), deep learning (TensorFlow, Keras, Pytorch), and natural language processing (NLTK, SpaCy, Gensim). For each category, it lists the top libraries and briefly describes their main functionalities. The document serves as an introduction to the Python data science ecosystem.

My Curriculum Vitae

adil raja

Muhammad Adil Raja is a researcher interested in machine learning and its applications. He has a PhD from University of Limerick and has worked as a post-doctorate researcher at Orange Labs. His research focuses on developing machine learning models for tasks like speech quality estimation, network impairment characterization, and computational neuroscience. He has extensive experience developing machine learning software and has authored several research proposals applying machine learning.

Week2: Programming for Data Analysis

Ferdin Joe John Joseph PhD

This document discusses merging and concatenating data in pandas. It describes how to use the concat() function to combine data from multiple sources like CSV, JSON, and XML files into a single dataframe. The syntax of concat() is explained and examples are provided of merging different arrays and dataframes, setting axes, performing inner and outer joins, and appending rows. Methods like append() are also covered as alternatives to concat() for combining two dataframes.

What's hot

Programming for Data Analysis: Week 3

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 5 - Cryptography and Steganography

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 2 - Blockchain Terminologies

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 3 - FinTech and Cryptocurrencies

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 4 - Hyperledger and Smart Contracts

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 1 - Introduction to Blockchain

Ferdin Joe John Joseph PhD

Blockchain Technology - Week 10 - CAP Teorem, Byzantines General Problem

Ferdin Joe John Joseph PhD

Programming for Data Analysis: Week 4

Ferdin Joe John Joseph PhD

Week 8: Programming for Data Analysis

Ferdin Joe John Joseph PhD

Week 10: Programming for Data Analysis

Ferdin Joe John Joseph PhD

Python Tutorial For Beginners | Python Crash Course - Python Programming Lang...

Edureka!

MongoDB and BlockChain

Massimo Brignoli

Blockchain .pptx

ManishKumar7730

Python vs rust

Simplilearn

Web3.0の話をしよう！ (NFT編).pdf

AtsushiSasaki9

ブロックチェーンを学ぶ　公開版

YamashitaKatsushi

Text mining

Pankaj Thakur

Introdução às criptomoedas: criando a sua própria moeda como o Bitcoin!

Alex Camargo

C MCQ

amutharajan10

Best Python Libraries For Data Science & Machine Learning | Edureka

Edureka!

What's hot (20)

Programming for Data Analysis: Week 3

Blockchain Technology - Week 5 - Cryptography and Steganography

Blockchain Technology - Week 2 - Blockchain Terminologies

Blockchain Technology - Week 3 - FinTech and Cryptocurrencies

Blockchain Technology - Week 4 - Hyperledger and Smart Contracts

Blockchain Technology - Week 1 - Introduction to Blockchain

Blockchain Technology - Week 10 - CAP Teorem, Byzantines General Problem

Programming for Data Analysis: Week 4

Week 8: Programming for Data Analysis

Week 10: Programming for Data Analysis

Python Tutorial For Beginners | Python Crash Course - Python Programming Lang...

MongoDB and BlockChain

Blockchain .pptx

Python vs rust

Web3.0の話をしよう！ (NFT編).pdf

ブロックチェーンを学ぶ　公開版

Text mining

Introdução às criptomoedas: criando a sua própria moeda como o Bitcoin!

C MCQ

Best Python Libraries For Data Science & Machine Learning | Edureka

Similar to Data wrangling week2

My Curriculum Vitae

adil raja

Week2: Programming for Data Analysis

Ferdin Joe John Joseph PhD

intro to python.pptx

UpasnaSharma37

Python is a popular programming language created in 1991 by Guido van Rossum. It can be used for web development, software development, mathematics, and system scripting. Python code can be executed immediately as it is written due to its interpreter system, allowing for quick prototyping. It works across different platforms and has a simple, English-like syntax. Common data types in Python include numeric, string, list, and tuple types.

An Introduction to Python Programming

Morteza Zakeri

This document provides an introduction to the Python programming language. It begins with an overview of Python's history and development timeline. It then discusses Python's key features such as being high-level, interpreted, object-oriented, and having extensive libraries. The document outlines Python's main programming paradigms and discusses its growing popularity. It provides details on getting started with Python, different integrated development environments (IDEs), and Python basics like variables, data types, operators, and input/output. The document also covers Python syntax and structures such as control flow, functions, classes, and exceptions handling.

Software Programming with Python II.pptx

GevitaChinnaiah

Data wrangling week 5

Ferdin Joe John Joseph PhD

This document outlines a lesson on JSON (JavaScript Object Notation). It introduces JSON, describing its basic syntax and components like objects, arrays, values, strings and whitespace. It demonstrates how to parse and work with JSON data in Python, including reading and writing JSON from files and URLs, converting between dictionaries and JSON, and appending data. As an activity, students are asked to analyze COVID-19 case data from an API in JSON format, wrangle the data, and submit their Python code. The next lesson materials will be available on the professor's GitHub page.

report on internshala python training

surabhimalviya1

The document is an internship report submitted by Debarati Banik to the Department of Computer Science and Engineering at the Institute of Engineering and Technology in Lucknow, India. It details Banik's participation in a summer 2022 internship with Internshala Trainings, where they learned Python programming, SQLite database connectivity, GUI development with PyQt, and applications of Python. The report includes acknowledgments, descriptions of the training modules, daily activities, problem analysis of a sample cricket team management game, coding examples, and bibliography.

Python indroduction

FEG

This document provides an introduction to Python programming basics for beginners. It discusses Python features like being easy to learn and cross-platform. It covers basic Python concepts like variables, data types, operators, conditional statements, loops, functions, OOPs, strings and built-in data structures like lists, tuples, and dictionaries. The document provides examples of using these concepts and recommends Python tutorials, third-party libraries, and gives homework assignments on using functions like range and generators.

Deep Learning with TensorFlow

Justin Grammens

I help companies to leverage the power of Deep Learning today. We review what Deep Learning is, how TensorFlow can be used in real world applications and some of the ways in which you can tune it to achieve the results that you wish. Contact me to learn more about our services at Lab651 where we help companies acquire data from the physical world and at Recursive Awesome where we perform Machine Learning and Analytics on the data to help you create better services and products for your customers.

hpcpp.pptx

pradhyumnpurohit1

Deep learning libraries TensorFlow and PyTorch are commonly used for machine learning. TensorFlow was developed by Google and has a faster compilation time than Keras or PyTorch. It supports CPUs and GPUs and uses data flow graphs with nodes and edges. PyTorch was originally developed as a Python wrapper for Torch and is pythonic in nature with dynamic computation graphs. Both support tensor computations and automatic differentiation, with PyTorch having richer APIs but fewer built-in tools than TensorFlow.

Programming with Python: Week 1

Ahmet Bulut

TeelTech - Advancing Mobile Device Forensics (online version)

Mike Felch

This document provides an overview of a training on advancing mobile device forensics through reverse engineering and programming techniques. It discusses how traditional forensic tools are becoming less effective at recovering data from newer devices and applications that are designed for privacy. The training will demonstrate extracting artifacts from a raw device image using a hex editor and Python scripts. It also outlines a simulated criminal investigation involving the murder of a victim, and how analyzing the digital evidence from the victim and suspect's mobile phones through these new techniques revealed deleted messages that are relevant to the case.

Analysis Mechanical system using Artificial intelligence

anishahmadgrd222

Machine learning from software developers point of view

Pierre Paci

Sahil Grover

Sahil Grover is a final year undergraduate student studying Computer Science and Engineering at IIT Kanpur. He has a strong academic record and has received several awards and honors. His skills include proficiency in languages like C++, JavaScript, Python, and tools like Git. He has experience with projects involving machine learning, compilers, and operating systems. He also has extensive achievements in competitive programming competitions.

presentation on data science with python

Khushbujaim

The document discusses a seminar on data science with Python. It defines data science as using insights from data to help business leaders make better decisions. Python is described as a popular and free tool for data analysis, visualization, and machine learning tasks in data science. The seminar covers Python basics like data types, looping, and libraries like NumPy for mathematics and Pandas for working with table-like data. Real-world applications of data science are also mentioned.

Data Structures and Algorithm - Week 4 - Trees, Binary Trees

Ferdin Joe John Joseph PhD

14_Ed_Symp_Open_Source

Steve Arnold

This document provides an overview of open source technologies that can be used in science education. It discusses definitions of free and open source software, examples of open source operating systems and applications, and specific tools for tasks like numerical computing, data analysis and visualization. The document also shares the author's experiences using open source technologies like Gentoo Linux, and concludes that open source is well-suited for education due to benefits like low costs, reliability and allowing students to examine internal workings.

Python programming ppt.pptx

nagendrasai12

Python is a versatile and widely-used high-level programming language known for its simplicity, readability, and extensive library support. Created by Guido van Rossum and first released in 1991, Python has since gained immense popularity across various domains, including web development, data science, scientific computing, artificial intelligence, and more. In this comprehensive description, we'll delve into Python's history, features, applications, and its vibrant community, highlighting why it continues to be a preferred choice for developers worldwide. Table of Contents Introduction to Python Python's History and Evolution Python's Key Features 3.1. Readability and Simplicity 3.2. High-level Language 3.3. Interpreted and Dynamic 3.4. Cross-platform Compatibility 3.5. Rich Standard Library 3.6. Community Support Python's Application Domains 4.1. Web Development 4.2. Data Science and Machine Learning 4.3. Scientific Computing 4.4. Automation and Scripting 4.5. Game Development 4.6. Desktop Applications Python Development Environments 5.1. IDLE 5.2. PyCharm 5.3. Jupyter Notebook 5.4. Visual Studio Code Getting Started with Python 6.1. Installing Python 6.2. Your First Python Program Python Syntax and Basic Concepts 7.1. Variables and Data Types 7.2. Conditional Statements 7.3. Loops 7.4. Functions 7.5. Exception Handling Working with Python Libraries 8.1. NumPy 8.2. Pandas 8.3. Matplotlib 8.4. Scikit-Learn Python and Web Development 9.1. Frameworks (Django, Flask) 9.2. Front-end Integration (HTML/CSS) 9.3. Database Interaction (SQL, NoSQL) Python in Data Science 10.1. Data Analysis with Pandas 10.2. Data Visualization with Matplotlib and Seaborn 10.3. Machine Learning with Scikit-Learn 10.4. Deep Learning with TensorFlow and PyTorch Scientific Computing with Python 11.1. Scientific Libraries (SciPy, SymPy) 11.2. Plotting and Visualization (Matplotlib) Automation and Scripting 12.1. Automating Tasks 12.2. Scripting for System Administration Game Development with Python 13.1. Pygame 13.2. Unity and Unreal Engine Integration Desktop Applications with Python 14.1. Tkinter 14.2. PyQt Python's Ecosystem and Package Management Python Best Practices 16.1. Code Readability (PEP 8) 16.2. Documentation and Comments 16.3. Testing (Unit Testing, pytest) 16.4. Version Control (Git) Python's Future and Trends Conclusion 1. Introduction to Python Python is a general-purpose, high-level programming language that was designed with a focus on code readability and simplicity. It uses an elegant and straightforward syntax that makes it easy for developers to express their ideas effectively, reducing the cost of program maintenance. Python's philosophy emphasizes the importance of code clarity and readability, which is encapsulated in the Zen of Python (PEP 20). The language has gained immense popularity due to its versatility and a rich ecosystem of libraries and frameworks. Python is renowned for its vibrant community and extensive documentation, making it in p

NLP and Deep Learning for non_experts

Sanghamitra Deb

Similar to Data wrangling week2 (20)

My Curriculum Vitae

Week2: Programming for Data Analysis

intro to python.pptx

An Introduction to Python Programming

Software Programming with Python II.pptx

Data wrangling week 5

report on internshala python training

Python indroduction

Deep Learning with TensorFlow

hpcpp.pptx

Programming with Python: Week 1

TeelTech - Advancing Mobile Device Forensics (online version)

Analysis Mechanical system using Artificial intelligence

Machine learning from software developers point of view

presentation on data science with python

Data Structures and Algorithm - Week 4 - Trees, Binary Trees

14_Ed_Symp_Open_Source

Python programming ppt.pptx

NLP and Deep Learning for non_experts

More from Ferdin Joe John Joseph PhD

Invited Talk DGTiCon 2022

Ferdin Joe John Joseph PhD

This document discusses blockchain and non-fungible tokens (NFTs). It provides examples of NFT use cases including digital art and assets. A case study of Bitscrunch, an NFT trading platform, is presented. Bitscrunch utilizes various technologies like AI and data analytics to provide services such as wash trade detection, price estimation, and portfolio tracking. NFT applications in education and the metaverse are also discussed. The talk concludes by emphasizing the growing role of virtual goods and digital ownership through NFTs.

Week 12: Cloud AI- DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

Cloud AI is a machine learning platform used to create and deploy machine learning models in the cloud. It allows users to streamline workflows including data preparation, model training, evaluation and publishing models for use in other cloud services. The platform provides services like Machine Learning Studio for visual modeling, Data Science Workshop for interactive development, and Elastic Algorithm Service to deploy models as APIs.

Week 11: Cloud Native- DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

Week 10: Cloud Security- DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

This document appears to be a slide deck presentation on cloud security given by Dr. Ferdin Joe John Joseph from the Faculty of Information Technology at the Thai-Nichi Institute of Technology in Bangkok. Over 42 slides, it likely covered various topics relating to security in cloud computing such as access controls, encryption, identity management, auditing, and compliance. The presentation concluded by noting that next week's topic will be on cloud native technologies, Kubernetes, DevOps, and include a demonstration.

Week 9: Relational Database Service Alibaba Cloud- DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

This document is a presentation on Relational Database Service (RDS) given by Dr. Ferdin Joe John Joseph from the Faculty of Information Technology at the Thai-Nichi Institute of Technology in Bangkok. The presentation introduces RDS as part of Alibaba Cloud's portfolio and provides a demonstration of its features and capabilities. It concludes by announcing that next week's topic will be on cloud security.

Week 7: Object Storage Service Alibaba Cloud- DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

This document discusses a lecture on object storage services from Alibaba Cloud. It provides an overview of Alibaba Cloud's portfolio and a demonstration of the Object Storage Service (OSS). The presentation was given by Dr. Ferdin Joe John Joseph from the Faculty of Information Technology at the Thai-Nichi Institute of Technology in Bangkok. Next week's lecture will cover relational database services and include another demonstration.

Week 6: Server Load Balancer and Auto Scaling Alibaba Cloud- DSA 441 Cloud Co...

Ferdin Joe John Joseph PhD

Week 5: Elastic Compute Service (ECS) with Alibaba Cloud- DSA 441 Cloud Compu...

Ferdin Joe John Joseph PhD

This document discusses a demonstration of Elastic Compute Service (ECS). It is presented by Dr. Ferdin Joe John Joseph from the Faculty of Information Technology at the Thai-Nichi Institute of Technology in Bangkok. The presentation covers ECS and is part of a cloud computing course. It provides an overview of ECS and previews that next week's topics will be autoscaling and server load balancing.

Week 4: Big Data and Hadoop in Alibaba Cloud - DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

Week 3: Virtual Private Cloud, On Premise, IaaS, PaaS, SaaS - DSA 441 Cloud C...

Ferdin Joe John Joseph PhD

This document summarizes a lecture on virtual private clouds, on-premise infrastructure versus cloud infrastructure models (IaaS, PaaS, SaaS), and the key differences between them. It discusses what a virtual private cloud (VPC) is, how it uses virtual switches (vSwitches) to connect resources, and some of its features. It then compares on-premise systems with the various cloud models in terms of control, scalability, costs and other factors. For each cloud model (IaaS, PaaS, SaaS), it provides definitions, architectural diagrams, benefits and examples. Lastly, it outlines some criteria to consider when choosing between the different cloud models.

Week 2: Virtualization and VM Ware - DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

Virtualization allows multiple virtual machines to run on a single physical server, improving resource utilization. It provides benefits like partitioning resources between VMs, portability by saving VMs as files, and security through hardware isolation. A hypervisor manages virtual resources and presents virtual machines to guest operating systems, allowing virtualization of CPUs, storage, and networks. Common hypervisors include VMware ESXi, Hyper-V, KVM, and Xen, which run directly on hardware or on a host operating system.

Week 1: Introduction to Cloud Computing - DSA 441 Cloud Computing

Ferdin Joe John Joseph PhD

This document outlines the syllabus for a 15-week cloud computing course. The course covers topics such as virtual machines, virtual private clouds, cloud services, elastic compute service, auto scaling, object storage, relational data service, cloud security, Kubernetes, and cloud platforms for AI. Students will complete assignments, a midterm exam, final exam, and capstone project. Assessment is based on attendance, midterm exam, assignments/project, and final exam.

Sept 6 2021 BTech Artificial Intelligence and Data Science curriculum

Ferdin Joe John Joseph PhD

The document outlines the program educational objectives, program outcomes, curriculum, and regulations for the B.Tech Artificial Intelligence and Data Science program at Anna University in Chennai, India. The 4-year program aims to provide students with proficiency in basic sciences, mathematics, AI, data science, and statistics to build data-driven systems. Students will develop technical skills to conduct research in AI and data science and create sustainable solutions. The curriculum covers topics such as data structures, algorithms, machine learning, deep learning, data analytics, and artificial intelligence across 8 semesters with theory, laboratory, and project components.

Hadoop in Alibaba Cloud

Ferdin Joe John Joseph PhD

Cloud Computing Essentials in Alibaba Cloud

Ferdin Joe John Joseph PhD

The document provides an overview of cloud computing essentials from Alibaba Cloud. It defines cloud computing according to NIST as enabling on-demand access to a shared pool of configurable computing resources. The speaker then discusses key cloud properties like scalability, availability, and fault tolerance. The presentation also provides details on Alibaba Cloud's global infrastructure and popular products like Elastic Compute Service, Server Load Balancing, Auto Scaling, and Object Storage Service. It concludes with information on cloud certification opportunities from Alibaba Cloud.

Transforming deep into transformers – a computer vision approach

Ferdin Joe John Joseph PhD

This keynote talk discusses how computer vision is transforming from traditional convolutional neural networks (CNNs) to vision transformers (ViTs). ViTs break images down into patches that are fed into a transformer encoder, similar to how text is handled with word embeddings. This approach performs competitively with CNNs while being conceptually simpler. The talk outlines the architecture of ViTs and how they function, noting they ignore convolutions and analyze variants' significance. It encourages attendees to start exploring ViTs through an online tutorial and contacts the speaker for additional help.

Week 11: Programming for Data Analysis

Ferdin Joe John Joseph PhD

The document discusses binary classification techniques including Naive Bayes classifiers and support vector machines (SVM). It explains that Naive Bayes classifiers use Bayes' theorem to calculate conditional probabilities for classification. Specifically, it describes how to calculate priors and fit probability distributions like binomial, multinomial, and Gaussian distributions for the Naive Bayes model. The document also outlines how SVMs find the optimal separating hyperplane between two classes by maximizing the margin between them. It provides code examples for implementing Naive Bayes and SVM classifiers and evaluating their performance.

Deep learning - Introduction

Ferdin Joe John Joseph PhD

This document discusses deep learning and convolutional neural networks (CNNs). It defines deep learning as a type of machine learning inspired by the human brain that uses artificial neural networks to learn representations of data. CNNs are described as a type of neural network commonly used for image and video recognition. The document outlines the basic architecture of CNNs including convolutional layers, pooling layers, and fully connected layers. It also discusses common CNN architectures like LeNet-5, AlexNet, VGG16 and VGG19.

Data wrangling week 11

Ferdin Joe John Joseph PhD

The document discusses web scraping and provides a demonstration. It defines web scraping as using software to automatically extract and organize useful data from websites. Originally screen scraping was used before the wide adoption of the World Wide Web, but as the Web grew, web scraping techniques were developed to automate the process. The demonstration shows how to use the Beautiful Soup library in Python to send requests to URLs, parse the HTML code, and access specific elements and content in tags. Students are assigned activities to practice scraping COVID-19 data from a website and analyzing a self-selected webpage.

Data wrangling week 9

Ferdin Joe John Joseph PhD

This document discusses data cleaning and handling dirty data. It introduces common data issues like missing values, typos, mixed formats, replicated entries, outliers, and violations of business rules. It then focuses on handling missing values and describes several techniques: listwise deletion, recovering values, educated guessing, average imputation, common-point imputation, regression substitution, and multiple imputation. The document also briefly mentions addressing typos, mixed formats, and replicated entries before assigning an activity to use pandas on CSV files to perform data cleaning.

More from Ferdin Joe John Joseph PhD (20)