Jeroen Janssens - Predicting at the Command Line

•

3 likes•912 views

PAPIs.io

Data & Analytics

Motivation Essential Tools and Concepts Use Case Conclusion
Let’s stay in touch
- www: datascienceatthecommandline.com
- email: jeroen@jeroenjanssens.com
- twitter: @jeroenhjanssens
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Overview
- Motivation
- Use Case
- Conclusion
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Motivation
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Data Science is OSEMN
- Obtaining data
- Scrubbing data
- Exploring data
- Modeling data
- iNterpreting data
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Command Line on Mac OS X
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Command Line on Ubuntu
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
The Command Line is Awesome
- Agile
- Augmenting
- Scalable
- Extensible
- Ubiquitous
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Essential Tools and
Concepts
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Command-line Tool is an Umbrella Term
- Binary executable
- Shell builtin
- Interpreted script
- Shell function
- Alias
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Concepts
- Executing a command-line tool
- Combining command-line tools
- Redirecting input and output
- Working with files
- Help!
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Over 80 Command-Line Tools
http://datascienceatthecommandline.com/#dst
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Most Important Tools for Tonight
- curl
- parallel
- jq
- csvkit
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Live demo
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Conclusion
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Three Pieces of Advice
- Be patient
- Be creative
- Be practical
Data Science at the Command Line Jeroen Janssens

Motivation Essential Tools and Concepts Use Case Conclusion
Happy hacking!
Data Science at the Command Line Jeroen Janssens

Similar to Jeroen Janssens - Predicting at the Command Line

Cloudera Breakfast Series, Analytics Part 1: Use All Your Data

Cloudera, Inc.

data-science-pdf-16588.pdf

vkharish18

San Francisco Hacker News - Machine Learning for Hackers

Adam Gibson

Speaker: Venkatesh Umaashankar LinkedIn: https://www.linkedin.com/in/venkateshumaashankar/ What will be discussed? What is Data Science? Types of data scientists What makes a Data Science Team? Who are its members? Why does a DS team need Full Stack Developer? Who should lead the DS Team Building a Data Science team in a Startup Vs Enterprise Case studies on: Evolution Of Airbnb’s DS Team How Facebook on-boards DS team and trains them Apple’s Acqui-hiring Strategy to build DS team Spotify -‘Center of Excellence’ Model Who should attend? Managers Technical Leaders who want to get started with Data Science

Building successful data science teams

Venkatesh Umaashankar

Tools: software eg databases, artificial intelligence and expert systems, predictive modelling; internet; others eg data mining systems Gather information: define the requirement; establish sources of information; define other factors to be considered eg constraints; select information Analyse information: quality eg validity, accuracy, currency, relevance; identify alternatives Management information: reports eg sales report, college enrolment statistics, marketing analysis (brick versus click)

Be able to use it tools to produce management system

Rajesh Khadka

Xpanse Analytics has developed a Predictive Modelling engine that automates the most time consuming tasks, typically conducted manually by Data Scientists, Business Analysts and Database Developers. These are the slides from a talk given by Xpanse’s founder, Maciek Wasiak, to the C will deliver a demo and answer any questions related to the Cork Big Data & Analytics Group. You can find out more about the company at www.xpanseanalytics.com

Xpanse Analytics Platform

Michael Keane

Data scientists face numerous challenges throughout the data science workflow that hinder productivity. As organizations continue to become more data-driven, a collaborative environment is more critical than ever — one that provides easier access and visibility into the data, reports and dashboards built against the data, reproducibility, and insights uncovered within the data.. Join us to hear how Databricks’ open and collaborative platform simplifies data science by enabling you to run all types of analytics workloads, from data preparation to exploratory analysis and predictive analytics, at scale — all on one unified platform.

Learn to Use Databricks for Data Science

Databricks

Data Scientist Salary, Skills, Jobs And Resume | Data Scientist Career | Data...

Simplilearn

Covid-19 has impacted operations for 78% of businesses, with 62% experiencing a fall in employee productivity. A key challenge for businesses is how to distribute information to their virtual digital workforces and customers seeking information online. In this webinar “How to Scale Information Dissemination for the Virtual Digital Workspace” we’ll explore: — How enterprise search empowers employees and customers, by achieving an unified single source of information; — Use cases where chatbots and smart question-answering systems can be deployed with the greatest effect. Full recording here: https://xenit.eu/webinars/virtual-digital-workspace/

How to Scale Information Dissemination to the Virtual Digital Workspace

XeniT Solutions nv

Slides from the Pivotal Open Source Hub Meetup "Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Science!" As the need for data science as a key differentiator grows in all industries, from large corporations to startups, the need to get to results quickly is enabled by sharing ideas and methods in the community. The data science team at Pivotal leverages and contributes to this community of publicly available and open source technologies as part of their practice. We will share the resources we use by highlighting specific toolkits for building models (e.g. MADlib, R) and visualization (e.g. Gephi and Circos) along with their benefits and limitations by sharing examples from Pivotal's data science engagements. At the end of this session we hope to have answered the questions: Where can I get started with Data Science? Which toolkit is most appropriate for building a model with my dataset? How can I visualize my results to have the greatest impact? Bio: Sarah Aerni is a member of the Pivotal Data Science team with a focus on healthcare and life science. She has a background in the field of Bioinformatics, developing tools to help biomedical researchers understand their data. She holds a B.S. In Biology with a specialization in Bioinformatics and minor in French Literature from UCSD, and an M.S. and Ph.D in Biomedical Informatics from Stanford University. During her time as a researcher she focused on the interface between machine learning and biology, building computational models enabling research for a broad range of fields in biomedicine. She also co-founded a start-up providing informatics services to researchers and small companies. At Pivotal she works with customers in life science and healthcare building models to derive insight and business value from their data.

Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Scienc...

Sarah Aerni

A Comprehensive Learning Path to Become a Data Science 2021.pptx

RajSingh512965

Data security is rapidly gaining importance as the volume of data companies collect, analyze and monetize grows exponentially. New data processing tools and platforms are emerging at an increasing rate, as are the ways in which an organization consumes data. In this presentation Mukund Sarma and Feni Chawla talk about the unique technical and cultural challenges of running a data security program and share some practical solutions that have worked well at our company. These slides were presented at the BSides Seattle 2024 conference.

BSides Seattle 2024 - Stopping Ethan Hunt From Taking Your Data.pptx

fenichawla

PPT5: Neuron Introduction

akira-ai

1645 track 2 pafka

Rising Media, Inc.

Esper ppt

VishalSingh1484

Expert System Seminar

Prof. Erwin Globio

"Empower Developers with HPE Machine Learning and Augmented Intelligence", Dr. Abdourahmane Faye, Big Data SME Lead DACH at HPE Watch more from Data Natives Berlin 2016 here: http://bit.ly/2fE1sEo Visit the conference website to learn more: www.datanatives.io Follow Data Natives: https://www.facebook.com/DataNatives https://twitter.com/DataNativesConf Stay Connected to Data Natives by Email: Subscribe to our newsletter to get the news first about Data Natives 2017: http://bit.ly/1WMJAqS About the Author: Abdou Faye is Subject Matter Expert in Big Data, Predictive Analytics / Machine Learning and Business Intelligence, with more than 19 years of experience in that area in various leading and executive roles, both from a Technical, Architecture and Sales perspectives. He recently joins HPE coming from SAP, where he was leading the Predictive Analysis & Big Data CoE (Center Of Excellence) business since 2010 for DACH, CEE and CIS region, in charge of Business Development and Sales Support. Prior to SAP, he worked 4 Years at Microsoft as Senior BI & SQL-Server Consultant in Switzerland, after 10 years spent at Philip Morris (CH), Orange Telco (CH) and SEMA Group (FR). Abdou graduated from Paris 11 University in 2000, where he completed a PhD on Data Mining/Predictive Analytics, after completing a Master in Computer Science.

"Empower Developers with HPE Machine Learning and Augmented Intelligence", Dr...

Dataconomy Media

Worldwide Intranet Challenge Intra Team Presentation V4

Deloitte Australia

Securitarian

Rupesh Verma

Adaptive Case Management Workshop 2014 - Keynote

Keith Swenson

Similar to Jeroen Janssens - Predicting at the Command Line (20)

Cloudera Breakfast Series, Analytics Part 1: Use All Your Data

data-science-pdf-16588.pdf

San Francisco Hacker News - Machine Learning for Hackers

Building successful data science teams

Be able to use it tools to produce management system

Xpanse Analytics Platform

Learn to Use Databricks for Data Science

Data Scientist Salary, Skills, Jobs And Resume | Data Scientist Career | Data...

How to Scale Information Dissemination to the Virtual Digital Workspace

Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Scienc...

A Comprehensive Learning Path to Become a Data Science 2021.pptx

BSides Seattle 2024 - Stopping Ethan Hunt From Taking Your Data.pptx

PPT5: Neuron Introduction

1645 track 2 pafka

Esper ppt

Expert System Seminar

"Empower Developers with HPE Machine Learning and Augmented Intelligence", Dr...

Worldwide Intranet Challenge Intra Team Presentation V4

Securitarian

Adaptive Case Management Workshop 2014 - Keynote

More from PAPIs.io

The daily job of a Data Scientist ranges from a variety of tasks: improving models performance or dealing with framework structure implementations. Machine learning as a service, a hot topic in the field, implies thinking about architecture to allow constant improvements in performance for our products. This presentation shows one architecture design using RESTful resources, document oriented databases and pre-trained pipelines to achieve real-time predictions of time series with high availability, scalability and freedom to Data Scientists work directly on improving the accuracy rate of our products. We fine tunned to work on time series forecasting which is a very challenging field that still needs better solutions in terms of innovative modeling. During the presentation will be shown how these decisions keep our Data Scientists focused on working with real data and thinking about improvements that can reach a large volume of time series instead of singular and localized actions.

Shortening the time from analysis to deployment with ml as-a-service — Luiz A...

PAPIs.io

Feature engineering is one of the most important, yet elusive, skills to master if you want to be a good data scientist. Machine learning competitions are hardly ever won with strong modeling techniques alone -- it is the combination of creative feature engineering and powerful modeling techniques that makes the difference. This tutorial will give the audience practical tips and tricks to improve the performance of machine learning algorithms. We will broadly look at feature engineering for applied machine learning, touching on subjects like: categorical vs. numerical variables, data cleaning, feature extraction, transformations, and imputation.

Feature engineering — HJ Van Veen (Nubank) @@PAPIs Connect — São Paulo 2017

PAPIs.io

For online businesses, recommender systems are paramount. There is an increasing need to take into account all the user information to tailor the best product offer, tailored to each new user. Part of that information is the content that the user actually sees: the visuals of the products. When it comes to products like luxury hotels, pictures of the room, the building or even the nearby beach can significantly impact users’ decision. In this talk, we will describe how we improved an online vacation retailer recommender system by using the information in images. We’ll explain how to leverage open data and pre-trained deep learning models to derive information on user taste. We will use a transfer learning approach that enables companies to use state of the art machine learning methods without needing deep learning expertise.

Extracting information from images using deep learning and transfer learning ...

PAPIs.io

Graphs are used to map relations on unstructured data. Companies’ data are most from database and mined using traditional data mining approach. However, model relational data as a graph can reveal useful insights and discovery relation among data that is ignored by traditional data mining techniques. In this work we used graphs to map physician relations using claim data as a proxy and this approach reveal interesting insights from health insurance company.

Discovering the hidden treasure of data using graph analytic — Ana Paula Appe...

PAPIs.io

Deep learning for sentiment analysis — André Barbosa (elo7) @PAPIs Connect — ...

PAPIs.io

Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...

PAPIs.io

In times of huge amounts of heterogeneous data available, processing and extracting knowledge requires more and more efforts on building complex software architectures. In this context, Apache Spark provides a powerful and efficient approach for large-scale data processing. This talk will briefly introduce a powerful machine learning library (MLlib) along with a general overview of the Spark framework, describing how to launch applications within a cluster. In this way, a demo will show how to simulate a Spark cluster in a local machine using images available on a Docker Hub public repository. In the end, another demo will show how to save time using unit tests for validating jobs before running them in a cluster.

Building machine learning applications locally with Spark — Joel Pinho Lucas ...

PAPIs.io

Battery life is critical for smart devices, but optimizing it requires cooperation from the entire software ecosystem. Wasteful software affects user perception about devices’ battery quality. Therefore, a large team within a producer of those smart devices is focused on identifying and correcting energy consumption bugs. Since the software ecosystem grows fast, that team faces a lot of suspect issues, from which only a small fraction turns out to be genuine. Our project aims to streamline energy-related bug processing in devices of the company and its partners, by automatically identifying anomalous behaviors related to battery drain using data mining and machine learning.

Battery log data mining — Ramon Oliveira (Datart) @PAPIs Connect — São Paulo ...

PAPIs.io

News recommendations are particularly challenging given the high number of new contents produced every day and the fast deterioration of its value for the users, demanding models and infrastructure able to deal with those nuances and serve a newly trained model about 100 times per day. Attending this presentation you're going to follow a detailed overview of how R&D team of Hearst's TV division is putting together Google BigQuery, Kubernetes cluster and Tensorflow to build a hybrid recommendation system combining model-based matrix factorization, content recency, and content semantics through NLP.

A tensorflow recommending system for news — Fabrício Vargas Matos (Hearst tv)...

PAPIs.io

Machine learning as a service (MLaS) is imperative to the success of many companies as many internal teams and organizations need to gain business intelligence from big data. Building a scalable MLaS in a very challenging problem. In this paper, we present the scalable MLaS we built for a company that operates globally. We focus on several scalability challenges and our technical solutions. Video at https://www.youtube.com/watch?v=MpnszJ_3Ong Couldn't attend PAPIs '16? Get access to the other presentations' slides and videos at https://gumroad.com/products/fehon/

Scaling machine learning as a service at Uber — Li Erran Li at #papis2016

PAPIs.io

This talk will offer answers to the following questions: What is data-driven decision making? What is AI? What is Business Intelligence? Why are these concepts important? What are the biggest challenges and opportunities? Daniel is the CEO of Satalia that provides AI inspired solutions to solve industries hardest problems. He’s the co-founder of the ASI that transitions scientists into data scientists. Daniel has a MSci and EngD in AI from UCL, and is Director of UCL’s Business Analytics MSc; applying AI to solve business/social problems. Daniel has many Advisory and Executive positions, holds an international Kauffman Global Entrepreneur Scholarship and actively promotes innovation across the globe.

Real-world applications of AI - Daniel Hulme @ PAPIs Connect

PAPIs.io

Possibly the most important lesson we have learned after 60 years of AI research is that what seemed to be very difficult to achieve, such as accurate medical diagnosis to playing chess at the level of a Grand Master, turned out to be relatively easy whereas what seemed easy, such as visual object recognition or deep language understanding, turned out to be extremely difficult. In my talk I will try to explain the reasons for this apparent contradiction by briefly reviewing the past and present of AI and projecting it into the near future. Ramon Lopez de Mantaras is Research Professor of the Spanish National Research Council (CSIC) and Director of the Artificial Intelligence Research Institute of the CSIC. Technical Engineer EE (Electrical Engineering) from the Technical Engineering School of Mondragón (Spain) in 1973. Master of Sciences in Automatic Control from the University of Toulouse III (France) in 1974, Ph.D. in Physics from the University of Toulouse III (France), in 1977, with a thesis in Robotics (done at LAAS, CNRS). Master of Science in Engineering (ComputerScience) from the University of California at Berkeley (USA) in 1979. Ph.D. in Computer Science, from the Technical University of Catalonia, Barcelona (Spain) in 1981.

Past, Present and Future of AI: a Fascinating Journey - Ramon Lopez de Mantar...

PAPIs.io

Everybody uses price promotions in retail. However, individual pricing is seldom used, particularly in offline retail. Marketing literature has been advocating the use of individual price discrimination for decades. Furthermore, product recommendations, ever-present in e-commerce, are also not often found in offline retail. We show the machine learning driven system behind a new promotion channel that enables retailers and manufacturers alike to target individual customers in offline retail. Lessons learned, technologies used, and machine learning approaches driving our system will be shown. Daniel Guhl has a background in economics & marketing, and got interested in data modeling during his Ph.D.. Currently, he is working as a data scientist at a Berlin based Start-up and is pursuing a postdoc at Humboldt University. He enjoys learning everyday and focuses on solving real world problems.

Revolutionizing Offline Retail Pricing & Promotions with ML - Daniel Guhl @ P...

PAPIs.io

Deep Learning (DL) is becoming a big tsunami in the Machine Learning community. This talk aims at introducing DL, its motivation and main techniques. However, part of this talk is also devoted to demystify DL. What are the main advantages but also the main drawbacks of DL?. And what are the key issues that the practitioners have to consider? Roberto Paredes is an Associate Professor at Departamento de Sistemas Informáticos y Computación DSIC of the Universidad Poliécnica de Valencia UPV. He belongs to the Pattern Recognition and Human Language Technologies Research Centre PRHLT. Roberto Paredes is the Director of the PRHLT and the President of the Spanish AERFAI Association. His main research interests are around the statistical learning, machine learning and more recently neural networks and deep learning.

Demystifying Deep Learning - Roberto Paredes Palacios @ PAPIs Connect

PAPIs.io

The best services have one thing in common: a superb customer experience. Banking services are no exception to this rule, and indeed the quest for an effortless, well informed, and personalized customer experience is one of the main goals of today's innovation in digital banking services. According to what Maslow has described in his "pyramid of needs", customers are seeking a more intimate and meaningful experience where banking services can actively assist the customer in performing and managing their financial life. Predictive APIs have a fundamental role in all this, as they enable a new set of customer journeys such as automatic categorization of transactions, detecting and alerting recurrent payments, pre-approving credit requests or provide better tools to fight fraud without limiting legitimate customer transactions. In this talk, I will focus on how to provide better banking services by using predictive APIs. I will describe the path on how to get there and the challenges of implementing predictive APIs in a strictly audited and regulated domain such as banking. Finally, I will briefly introduce a number of data science techniques to implement those customer journeys and describe how big/fast data engineering can be used to realize predictive data pipelines. Natalino is currently Enterprise Data Architect at ING in the Netherlands, where leads the strategy, definition, design and implementation of big/fast data solutions for data-driven applications, for personalized marketing, predictive analytics, and fraud/security management. All-round Software Architect, Data Technologist, Innovator, with 15+ years experience in research, development and management of distributed architectures and scalable services and applications.

Predictive APIs: What about Banking? - Natalino Busa @ PAPIs Connect

PAPIs.io

Fintech startups are taking business away from traditional institutions like banks, exchanges, and brokerages. One of the reasons that these startups are able to compete with $30B+ behemoths like Credit Suisse and Goldman Sachs is their advanced decision making capabilities. By leveraging new data sources and better predictive analytics, companies like Ferratum Bank can make more accurate decisions in a fraction of the time. This talk will cover: Types of decisions you can automate Challenges in building predictive, financial apps First-hand, real-world examples Greg Lamp is the co-Founder and CTO of Yhat. In this role, Greg leads development of Yhat's core products and infrastructure and is the principal architect of the company's cloud and on-premise enterprise software applications. Greg was previously a product manager at OnDeck, a fintech startup in New York and before that an analyst at comScore. Greg is a graduate of the University of Virginia.

Microdecision making in financial services - Greg Lamp @ PAPIs Connect

PAPIs.io

What is the future we want to create, and what can we do – starting today – to actively shape that future with general AI? This talk outlines a vision for the future of humankind once AI reaches human or superhuman levels, and leads the audience through the steps one research group is taking to get there. From the economics of smart robots and job replacement, to bionic humans exploring the universe through space travel, the talk offers a window into the work of 30 researchers focused on AI development and safety, and explains what attendees can do themselves to help make that future happen. JoEllen is the AI Safety Ambassador and Head of PR for GoodAI, a Prague-based general AI research and development company. A high school teacher by trade, she has a bachelor’s degrees in English and Philosophy from Seattle University, a master’s degree in Transatlantic Studies from Charles University in Prague, and is the recipient of Fulbright grant. JoEllen is particularly interested in how AI will affect international government and political relations.

Engineering the Future of Our Choice with General AI - JoEllen Lukavec Koeste...

PAPIs.io

Training deep networks is a time-consuming process, with networks for object recognition often requiring multiple days to train. For this reason, leveraging the resources of a cluster to speed up training is an important area of work. In this talk we'll show how to use an AWS Spark cluster to train a model quickly from a laptop at a very little cost (around 10€). Vincent Van Steenbergen is a freelance (big) data engineer who's working on a range of international projects, implementing systems able to handle terabytes of data, usually involving Spark, Scala, Kafka, Hadoop and Cassandra. His main interest right now is applying these techniques to solve machine learning problems. Vincent was previously a technical architect at Property. Works, a real estate startup in London and before that an R&D engineer at IDAaaS.

Distributed deep learning with spark on AWS - Vincent Van Steenbergen @ PAPIs...

PAPIs.io

Shopping, or as the people on the other side of the counter call it, retail has become the number one breeding ground for predictive applications in the enterprise. What started as simple recommendation engines has evolved into a complex and powerful ecosystem of predictive applications that affect core processes such as pricing, replenishment and staff planning. In this talk, Ulrich Kerzel will share impact and experiences from building and operating predictive applications for large retailers, and explain why the future of retail is as much a science as an art. Dr. Ulrich Kerzel is a Senior data scientists at Blue Yonder and renowned scientist with research experience at the University of Cambridge and CERN. Ulrich Kerzel earned his PhD under Professor Dr Feindt at the US Fermi National Laboratory and at that time made a considerable contribution to core technology of NeuroBayes. After his PhD, he went to the University of Cambridge, were he was a Senior Research Fellow at Magdelene College. His research work focused on complex statistical analyses to understand the origin of matter and antimatter using data from the LHCb experiment at the Large Hadron Collider at CERN, the world’s biggest research institute for particle physics. He continued this work as a Research Fellow at CERN before he came to Blue Yonder as a senior data scientist.

How to predict the future of shopping - Ulrich Kerzel @ PAPIs Connect

PAPIs.io

We live in a world of data, of big data. A big portion of this data has been generated by humans, and particularly through their mobile phones. In fact, there are almost as many mobile phones in the world as humans. The mobile phone is the piece of technology with the highest levels of adoption in human history. We carry them with us all through the day (and night, in many cases), leaving digital traces of our physical interactions. Mobile phones have become sensors of human activity in the large scale and also the most personal devices. In my talk, I will present some of the work that we are doing at Telefonica Research in the area of human behavior understanding from data captured with mobile phones, and particularly our work in the area of Big Data for Social Good. I will highlight opportunities but also challenges that we would need to address in order to truly leverage this opportunity. Nuria Oliver is a computer scientist and Scientific Director at Telefónica. She holds a Ph.D. from the Media Lab at MIT. She is one of the most cited female computer scientist in Spain, with her research having been cited by more than 8900 publications. She is well known for her work in computational models of human behavior, human computer-interaction, intelligent user interfaces, mobile computing and big data for social good.

The emergent opportunity of Big Data for Social Good - Nuria Oliver @ PAPIs C...

PAPIs.io

More from PAPIs.io (20)