This talk will focus on Techniques, metrics and different tests (code, models, infra and features/data) that help the developers of machine learning systems to achieve CD.
Barcelona Digital Festival 28th Nov 2019 - Data Analytics in eSports. UbeatCa...CIO Edge
Taken from our BCN Digital Festival last week, for info on attending, speaking or sponsoring our next event on the 29/30th April 2020 email enquiry@digitalenterprisefest.com
Data Analytics in eSports. UbeatCase Study
Building AI & Automate services need a solid base on Data Management, but the current environment is volatile, uncertain, complex and ambiguous so you never know what data will be important in the following months.
The Data Management Platform in an extremely dynamic market like eSports where everything is currently being created, in reinvention and is to be validated is even more challenging.
UBEAT is the leading streaming platform of eSports related content. Created in November 2018, it still hasn’t 12 months of existence but a lot of learnings in its rear mirror and a lot of future to come in its high beam. Especially regarding Data Management.
To apply to speak or sponsor our 2020 events goto www.digitalenterprisefest.com
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Codemotion
In this talk Gerbert will give an overview of Artificial Intelligence, outline the current state of the art in research and explain what it takes to actually do an AI project. Using practical cases and tools he will give you insight in the phases of an AI project and explain some of the problems you might encounter along the way and how you might be able to solve them.
A global leading design in the car video recorder industry, the world's smallest car hard disk video recorder, a patented technology with one-button hard disk quick installation and removal functions, which solves the trouble of hard disk installation and the inconvenience of disassembling and reading data. All-aluminum metal body, secondary oxidation process, brushed panel, laser engraving, and anti-misplug aviation interface. Our technical design team makes such a small volume equipped with the most advanced H.265 high-efficiency video coding technology, full HD 1080P video effect.
The exquisite appearance and patented technology ensure our originality of technology and the driving force behind our company's team. Technology is the foundation of a company, and professionalism is its mission.
Presented the 28th October 2015 at the 6th International Conference and Exhibition on body Scanning Technologies 2015, Hometrica Consulting, Lugano, Switzerland.
The access to the 3D representation of people’s body shape has multiple applications to consumer goods which performance is related to human body dimensions or shape. This is the case of wearables such as clothing, footwear, headgear, orthotics, or equipment/environments such as furniture, transports or workstations. Some of the existing and potential applications of 3D human representations include personalisation, virtual try-on or size allocation for wearables or product configuration/adjustment for equipment/environments.
However, the cost of 3D scanners is high; the devices are too bulky for homes and retail stores; and its proper use requires expertise to get the relevant parameters from the 3D object (e.g. measurements). These three barriers are currently hindering the massive spreading of 3D scanners as consumer good or as typical in-store appliance.
This paper describes an array of approaches for realistically estimating human 3D shapes (i.e. full bodies or feet) using a regular smartphone or just entering a set of parameters (e.g. age, gender and self-taken measurements). The proposed approaches are based on data-driven 3D reconstructions, using parameterised shape spaces created from large 3D human body or feet databases. The algorithm finds the combination of shape parameters that best matches either the silhouettes extracted from the images or the body measurements entered.
Despite not being actual body scanners, these solutions are easy-to-use and can provide enough accuracy for applications such as virtual try-on, made-to-measure or size allocation of certain types of wearables. Moreover, they can be distributed to the final consumer or to the points of sale at a really reduced cost (or even for free), thus overcoming the main barriers to the massive spreading of its use in e-commerce, new retail experiences, new production pipelines or new business models.
In order to illustrate these technologies, some examples of application to different contexts (i.e. virtual worlds, e-commerce and personalisation) are presented: virtual try-on of female fashion (VisuaLook), size allocation for childrenswear (KIDSIZE), personalised comfort insoles (Sunfeet) and personalised shoes (Feetz).
This talk will focus on Techniques, metrics and different tests (code, models, infra and features/data) that help the developers of machine learning systems to achieve CD.
Barcelona Digital Festival 28th Nov 2019 - Data Analytics in eSports. UbeatCa...CIO Edge
Taken from our BCN Digital Festival last week, for info on attending, speaking or sponsoring our next event on the 29/30th April 2020 email enquiry@digitalenterprisefest.com
Data Analytics in eSports. UbeatCase Study
Building AI & Automate services need a solid base on Data Management, but the current environment is volatile, uncertain, complex and ambiguous so you never know what data will be important in the following months.
The Data Management Platform in an extremely dynamic market like eSports where everything is currently being created, in reinvention and is to be validated is even more challenging.
UBEAT is the leading streaming platform of eSports related content. Created in November 2018, it still hasn’t 12 months of existence but a lot of learnings in its rear mirror and a lot of future to come in its high beam. Especially regarding Data Management.
To apply to speak or sponsor our 2020 events goto www.digitalenterprisefest.com
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Codemotion
In this talk Gerbert will give an overview of Artificial Intelligence, outline the current state of the art in research and explain what it takes to actually do an AI project. Using practical cases and tools he will give you insight in the phases of an AI project and explain some of the problems you might encounter along the way and how you might be able to solve them.
A global leading design in the car video recorder industry, the world's smallest car hard disk video recorder, a patented technology with one-button hard disk quick installation and removal functions, which solves the trouble of hard disk installation and the inconvenience of disassembling and reading data. All-aluminum metal body, secondary oxidation process, brushed panel, laser engraving, and anti-misplug aviation interface. Our technical design team makes such a small volume equipped with the most advanced H.265 high-efficiency video coding technology, full HD 1080P video effect.
The exquisite appearance and patented technology ensure our originality of technology and the driving force behind our company's team. Technology is the foundation of a company, and professionalism is its mission.
Presented the 28th October 2015 at the 6th International Conference and Exhibition on body Scanning Technologies 2015, Hometrica Consulting, Lugano, Switzerland.
The access to the 3D representation of people’s body shape has multiple applications to consumer goods which performance is related to human body dimensions or shape. This is the case of wearables such as clothing, footwear, headgear, orthotics, or equipment/environments such as furniture, transports or workstations. Some of the existing and potential applications of 3D human representations include personalisation, virtual try-on or size allocation for wearables or product configuration/adjustment for equipment/environments.
However, the cost of 3D scanners is high; the devices are too bulky for homes and retail stores; and its proper use requires expertise to get the relevant parameters from the 3D object (e.g. measurements). These three barriers are currently hindering the massive spreading of 3D scanners as consumer good or as typical in-store appliance.
This paper describes an array of approaches for realistically estimating human 3D shapes (i.e. full bodies or feet) using a regular smartphone or just entering a set of parameters (e.g. age, gender and self-taken measurements). The proposed approaches are based on data-driven 3D reconstructions, using parameterised shape spaces created from large 3D human body or feet databases. The algorithm finds the combination of shape parameters that best matches either the silhouettes extracted from the images or the body measurements entered.
Despite not being actual body scanners, these solutions are easy-to-use and can provide enough accuracy for applications such as virtual try-on, made-to-measure or size allocation of certain types of wearables. Moreover, they can be distributed to the final consumer or to the points of sale at a really reduced cost (or even for free), thus overcoming the main barriers to the massive spreading of its use in e-commerce, new retail experiences, new production pipelines or new business models.
In order to illustrate these technologies, some examples of application to different contexts (i.e. virtual worlds, e-commerce and personalisation) are presented: virtual try-on of female fashion (VisuaLook), size allocation for childrenswear (KIDSIZE), personalised comfort insoles (Sunfeet) and personalised shoes (Feetz).
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014Austin Ogilvie
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
-----------
Slides from a talk by Greg Lamp, CTO of Yhat, about building recommendation systems using Python and deploying them to production.
Speaker: Umayah Abdennabi
Agenda
* Intro Grammarly (Umayah Abdennabi, 5 mins)
* Meetup Updates and Announcements (Chris, 5 mins)
* Custom Functions in Spark SQL (30 mins)
Speaker: Umayah Abdennabi
Spark comes with a rich Expression library that can be extended to make custom expressions. We will look into custom expressions and why you would want to use them.
* TF 2.0 + Keras (30 mins)
Speaker: Francesco Mosconi
Tensorflow 2.0 was announced at the March TF Dev Summit, and it brings many changes and upgrades. The most significant change is the inclusion of Keras as the default model building API. In this talk, we'll review the main changes introduced in TF 2.0 and highlight the differences between open source Keras and tf.keras
* SQUAD Deep-Dive: Question & Answer with Context (45 mins)
Speaker: Brett Koonce (https://quarkworks.co)
SQuAD (Stanford Question Answer Dataset) is an NLP challenge based around answering questions by reading Wikipedia articles, designed to be a real-world machine learning benchmark. We will look at several different ways to tackle the SQuAD problem, building up to state of the art approaches in terms of time, complexity, and accuracy.
https://rajpurkar.github.io/SQuAD-explorer/
https://dawn.cs.stanford.edu/benchmark/#squad
Food and drinks will be provided. The event will be held at Grammarly's office at One Embarcadero Center on the 9th floor. When you arrive at One Embarcadero, take the escalator to the second floor where you will find the lobby and elevators to the office suites. Come on up to the 9th floor (no need to check in at security), and ring the Grammarly doorbell.
Scaling Ride-Hailing with Machine Learning on MLflowDatabricks
"GOJEK, the Southeast Asian super-app, has seen an explosive growth in both users and data over the past three years. Today the technology startup uses big data powered machine learning to inform decision-making in its ride-hailing, lifestyle, logistics, food delivery, and payment products. From selecting the right driver to dispatch, to dynamically setting prices, to serving food recommendations, to forecasting real-world events. Hundreds of millions of orders per month, across 18 products, are all driven by machine learning.
Building production grade machine learning systems at GOJEK wasn't always easy. Data processing and machine learning pipelines were brittle, long running, and had low reproducibility. Models and experiments were difficult to track, which led to downstream problems in production during serving and model evaluation. In this talk we will cover these and other challenges that we faced while trying to scale end-to-end machine learning systems at GOJEK. We will then introduce MLflow and explore the key features that make it useful as part of an ML platform. Finally, we will show how introducing MLflow into the ML life cycle has helped to solve many of the problems we faced while scaling machine learning at GOJEK.
"
Data Science in Production: Technologies That Drive Adoption of Data Science ...Nir Yungster
Critical to a data science team’s ability to drive impact is its effectiveness in incorporating its solutions into new or existing products. When collaborating with other engineering teams, and especially when solutions must operate at scale, technological choices can be critical factors in determining what type of outcome you'll have. We walk through strategies and specific technologies - Airflow, Docker, Kubernetes - that can help promote successful collaboration between data science and engineering.
Building a Beer Recommender with Yhat (PAPIs.io - November 2014)Austin Ogilvie
Building the predictive aspect of applications is the fun, sexy part. New tools like scikit-learn, pandas, and R have made building models less painful, but deploying/embedding models into production applications is challenging. We'll show how Yhat makes deploying predictive models written in Python or R fast and easy by building a beer recommendation system and an accompanying webapp.
[DSC Europe 22] What is Audio Data Augmentation? Techniques, Best Practices, ...DataScienceConferenc1
Insufficient data is one of the most common challenges in implementing machine learning and deep learning in particular. Collecting data can be costly and time-consuming, especially in the case of audio data. This presentation will cover different audio data augmentation techniques, the theory behind them, their implementation, best practices, and finally, what benefits they can bring.
Graph Gurus Episode 26: Using Graph Algorithms for Advanced Analytics Part 1TigerGraph
Full Webinar: https://info.tigergraph.com/graph-gurus-26
Have you ever wondered how routing apps like Google Maps find the best route from one place to another? Finding that route is solved by the Shortest Path graph algorithm. Today, graph algorithms are moving from the classroom to a host of important and valuable operational and analytical applications. This webinar will give you an overview of graph algorithms, how to use them, and the categories of problems they can solve, and then take a closer look at path algorithms. This webinar is the first part in a five-part series, each part examining a different type of problem to be solved.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2lGNybu.
Stefan Krawczyk discusses how his team at StitchFix use the cloud to enable over 80 data scientists to be productive. He also talks about prototyping ideas, algorithms and analyses, how they set up & keep schemas in sync between Hive, Presto, Redshift & Spark and make access easy for their data scientists, etc. Filmed at qconsf.com..
Stefan Krawczyk is Algo Dev Platform Lead at StitchFix, where he’s leading development of the algorithm development platform. He spent formative years at Stanford, LinkedIn, Nextdoor & Idibon, working on everything from growth engineering, product engineering, data engineering, to recommendation systems, NLP, data science and business intelligence.
In this talk we’ll present the technology behind the Fully Automated Store by Checkout Technologies. The actual version of the store is a result of the work of 12 engineers that spans the areas from hardware and design to the ultimate deep learning architectures. Will be also discussed the challenges and lessons learnt during this adventure and what it means to deploy the system which has an AI engine in its core. Creation of the dataset and the invention of the specific metrics that is capable to measure the accuracy of the entire system will be discussed.
LeddarVu8: The first off-the-shelf solid state high-definition LiDAR module f...Yole Developpement
LeddarTech’s new LiDAR has no moving parts, giving it the smallest form factor while integrating the latest innovations in LiDAR technology
The light detection and ranging principle and technology, also known as LiDAR, is used in a wide spectrum of applications. With upcoming autonomous or “self-driving” cars, LiDAR technology is being considered for one option for the “eyes” of such vehicles. However, existing systems do not match the challenging reliability, compactness and cost-efficiency specifications of the automotive world.
Based on “LeddarCore” components, Leddartech has developed LeddarVu8, a compact solid-state LiDAR without mechanical movement that provides highly accurate multi-target detection over eight independent segments. It detects targets at up to 215 m range despite its tiny size of 70 mm x 35 mm x 46 mm and weighing only 75 grams. The LeddarVu8 delivers nearly twice the range for half the volume compared to the previous version.
These qualities qualify the LeddarVu8 as a candidate for advanced driver assistance, replacing the radar. With its adjustable and interchangeable optics module, the LeddarVu8 can also be interesting for other applications, such as traffic management, speed enforcement, heavy-duty equipment collision warning, navigation, liquid level sensing and bulk volume measurements.
The light is emitted by Excelitas’ tri-stacked emitting edge laser diode of 75W and received by a photodiode array. The two components are designed for large volume markets and some innovations reduce their usually high cost. This first automotive version of the LeddarVu8 shows the full potential of this technology to go under the $100 barrier.
This report presents the full system manufacturing and packaging processes for the LeddarVu8 as well as the physical analysis of the two main optical components, the laser diode and the photodiode. An estimation of the manufacturing cost and selling price is included.
More information on that report at http://www.i-micronews.com/reports.html
WSO2Con USA 2015: An Introduction to the WSO2 Analytics PlatformWSO2
In today’s connected world organizations have access to an enormous amount of data. We often don’t know what they mean or how we can use them, in terms of hindsight, oversight, insight and foresight, to gain competitive advantage in the market. Use cases ranging from simple system monitoring to complex fraud analysis demands this.
The WSO2 Data Analytics platform lets you collect data, allows you to explore it through batch, real-time, interactive and predictive processing technologies and allows you to communicate your results. In this talk, we will discuss the WSO2 Data Analytics platform and how it brings together all analytics technologies into a single platform and user experience.
What uses for observing operations of Configuration Management?RUDDER
Nicolas Charles, CfgMgmtCamp 2019.
More and more services expose their state, internal details and metrics to be observable, and improve overall quality of service.
But what about observing the infrastructure they are deployed, configured and maintained on?
What can we learn from that, and what do we need from configuration management to get these features and metrics?
Logs from installation is a good start, but they need centralization, aggregation and especially knowledge derivation from these - but also we need to observe these features over time, to trace changes, and correlate them with monitoring.
Rudder was built around the predicate that all actions of the configuration agent need to be traced, centralized and exposed in a meaningful way - with agents ensuring the continuous configuration of systems, and this talk will show the rationale behind this predicate, how we implemented this solution, and the benefits of this approach for the modern IT world.
MLOps vs LLMOps (by workflows and use cases) - 2024-05-21Alessandra Bilardi
MLOps @ localhost 2024
A pragmatic approach to manage ML systems by workflows and use cases.
https://www.grusp.org/conferenze_/mlops-127-0-0-1-21-maggio-2024/
More Related Content
Similar to The Fourier transformation - 2023-07-23
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014Austin Ogilvie
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
-----------
Slides from a talk by Greg Lamp, CTO of Yhat, about building recommendation systems using Python and deploying them to production.
Speaker: Umayah Abdennabi
Agenda
* Intro Grammarly (Umayah Abdennabi, 5 mins)
* Meetup Updates and Announcements (Chris, 5 mins)
* Custom Functions in Spark SQL (30 mins)
Speaker: Umayah Abdennabi
Spark comes with a rich Expression library that can be extended to make custom expressions. We will look into custom expressions and why you would want to use them.
* TF 2.0 + Keras (30 mins)
Speaker: Francesco Mosconi
Tensorflow 2.0 was announced at the March TF Dev Summit, and it brings many changes and upgrades. The most significant change is the inclusion of Keras as the default model building API. In this talk, we'll review the main changes introduced in TF 2.0 and highlight the differences between open source Keras and tf.keras
* SQUAD Deep-Dive: Question & Answer with Context (45 mins)
Speaker: Brett Koonce (https://quarkworks.co)
SQuAD (Stanford Question Answer Dataset) is an NLP challenge based around answering questions by reading Wikipedia articles, designed to be a real-world machine learning benchmark. We will look at several different ways to tackle the SQuAD problem, building up to state of the art approaches in terms of time, complexity, and accuracy.
https://rajpurkar.github.io/SQuAD-explorer/
https://dawn.cs.stanford.edu/benchmark/#squad
Food and drinks will be provided. The event will be held at Grammarly's office at One Embarcadero Center on the 9th floor. When you arrive at One Embarcadero, take the escalator to the second floor where you will find the lobby and elevators to the office suites. Come on up to the 9th floor (no need to check in at security), and ring the Grammarly doorbell.
Scaling Ride-Hailing with Machine Learning on MLflowDatabricks
"GOJEK, the Southeast Asian super-app, has seen an explosive growth in both users and data over the past three years. Today the technology startup uses big data powered machine learning to inform decision-making in its ride-hailing, lifestyle, logistics, food delivery, and payment products. From selecting the right driver to dispatch, to dynamically setting prices, to serving food recommendations, to forecasting real-world events. Hundreds of millions of orders per month, across 18 products, are all driven by machine learning.
Building production grade machine learning systems at GOJEK wasn't always easy. Data processing and machine learning pipelines were brittle, long running, and had low reproducibility. Models and experiments were difficult to track, which led to downstream problems in production during serving and model evaluation. In this talk we will cover these and other challenges that we faced while trying to scale end-to-end machine learning systems at GOJEK. We will then introduce MLflow and explore the key features that make it useful as part of an ML platform. Finally, we will show how introducing MLflow into the ML life cycle has helped to solve many of the problems we faced while scaling machine learning at GOJEK.
"
Data Science in Production: Technologies That Drive Adoption of Data Science ...Nir Yungster
Critical to a data science team’s ability to drive impact is its effectiveness in incorporating its solutions into new or existing products. When collaborating with other engineering teams, and especially when solutions must operate at scale, technological choices can be critical factors in determining what type of outcome you'll have. We walk through strategies and specific technologies - Airflow, Docker, Kubernetes - that can help promote successful collaboration between data science and engineering.
Building a Beer Recommender with Yhat (PAPIs.io - November 2014)Austin Ogilvie
Building the predictive aspect of applications is the fun, sexy part. New tools like scikit-learn, pandas, and R have made building models less painful, but deploying/embedding models into production applications is challenging. We'll show how Yhat makes deploying predictive models written in Python or R fast and easy by building a beer recommendation system and an accompanying webapp.
[DSC Europe 22] What is Audio Data Augmentation? Techniques, Best Practices, ...DataScienceConferenc1
Insufficient data is one of the most common challenges in implementing machine learning and deep learning in particular. Collecting data can be costly and time-consuming, especially in the case of audio data. This presentation will cover different audio data augmentation techniques, the theory behind them, their implementation, best practices, and finally, what benefits they can bring.
Graph Gurus Episode 26: Using Graph Algorithms for Advanced Analytics Part 1TigerGraph
Full Webinar: https://info.tigergraph.com/graph-gurus-26
Have you ever wondered how routing apps like Google Maps find the best route from one place to another? Finding that route is solved by the Shortest Path graph algorithm. Today, graph algorithms are moving from the classroom to a host of important and valuable operational and analytical applications. This webinar will give you an overview of graph algorithms, how to use them, and the categories of problems they can solve, and then take a closer look at path algorithms. This webinar is the first part in a five-part series, each part examining a different type of problem to be solved.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2lGNybu.
Stefan Krawczyk discusses how his team at StitchFix use the cloud to enable over 80 data scientists to be productive. He also talks about prototyping ideas, algorithms and analyses, how they set up & keep schemas in sync between Hive, Presto, Redshift & Spark and make access easy for their data scientists, etc. Filmed at qconsf.com..
Stefan Krawczyk is Algo Dev Platform Lead at StitchFix, where he’s leading development of the algorithm development platform. He spent formative years at Stanford, LinkedIn, Nextdoor & Idibon, working on everything from growth engineering, product engineering, data engineering, to recommendation systems, NLP, data science and business intelligence.
In this talk we’ll present the technology behind the Fully Automated Store by Checkout Technologies. The actual version of the store is a result of the work of 12 engineers that spans the areas from hardware and design to the ultimate deep learning architectures. Will be also discussed the challenges and lessons learnt during this adventure and what it means to deploy the system which has an AI engine in its core. Creation of the dataset and the invention of the specific metrics that is capable to measure the accuracy of the entire system will be discussed.
LeddarVu8: The first off-the-shelf solid state high-definition LiDAR module f...Yole Developpement
LeddarTech’s new LiDAR has no moving parts, giving it the smallest form factor while integrating the latest innovations in LiDAR technology
The light detection and ranging principle and technology, also known as LiDAR, is used in a wide spectrum of applications. With upcoming autonomous or “self-driving” cars, LiDAR technology is being considered for one option for the “eyes” of such vehicles. However, existing systems do not match the challenging reliability, compactness and cost-efficiency specifications of the automotive world.
Based on “LeddarCore” components, Leddartech has developed LeddarVu8, a compact solid-state LiDAR without mechanical movement that provides highly accurate multi-target detection over eight independent segments. It detects targets at up to 215 m range despite its tiny size of 70 mm x 35 mm x 46 mm and weighing only 75 grams. The LeddarVu8 delivers nearly twice the range for half the volume compared to the previous version.
These qualities qualify the LeddarVu8 as a candidate for advanced driver assistance, replacing the radar. With its adjustable and interchangeable optics module, the LeddarVu8 can also be interesting for other applications, such as traffic management, speed enforcement, heavy-duty equipment collision warning, navigation, liquid level sensing and bulk volume measurements.
The light is emitted by Excelitas’ tri-stacked emitting edge laser diode of 75W and received by a photodiode array. The two components are designed for large volume markets and some innovations reduce their usually high cost. This first automotive version of the LeddarVu8 shows the full potential of this technology to go under the $100 barrier.
This report presents the full system manufacturing and packaging processes for the LeddarVu8 as well as the physical analysis of the two main optical components, the laser diode and the photodiode. An estimation of the manufacturing cost and selling price is included.
More information on that report at http://www.i-micronews.com/reports.html
WSO2Con USA 2015: An Introduction to the WSO2 Analytics PlatformWSO2
In today’s connected world organizations have access to an enormous amount of data. We often don’t know what they mean or how we can use them, in terms of hindsight, oversight, insight and foresight, to gain competitive advantage in the market. Use cases ranging from simple system monitoring to complex fraud analysis demands this.
The WSO2 Data Analytics platform lets you collect data, allows you to explore it through batch, real-time, interactive and predictive processing technologies and allows you to communicate your results. In this talk, we will discuss the WSO2 Data Analytics platform and how it brings together all analytics technologies into a single platform and user experience.
What uses for observing operations of Configuration Management?RUDDER
Nicolas Charles, CfgMgmtCamp 2019.
More and more services expose their state, internal details and metrics to be observable, and improve overall quality of service.
But what about observing the infrastructure they are deployed, configured and maintained on?
What can we learn from that, and what do we need from configuration management to get these features and metrics?
Logs from installation is a good start, but they need centralization, aggregation and especially knowledge derivation from these - but also we need to observe these features over time, to trace changes, and correlate them with monitoring.
Rudder was built around the predicate that all actions of the configuration agent need to be traced, centralized and exposed in a meaningful way - with agents ensuring the continuous configuration of systems, and this talk will show the rationale behind this predicate, how we implemented this solution, and the benefits of this approach for the modern IT world.
MLOps vs LLMOps (by workflows and use cases) - 2024-05-21Alessandra Bilardi
MLOps @ localhost 2024
A pragmatic approach to manage ML systems by workflows and use cases.
https://www.grusp.org/conferenze_/mlops-127-0-0-1-21-maggio-2024/
How to move your ML system from local to production - 2024-03-15Alessandra Bilardi
Incontro DevOps Italia 2024
When a Cloud Engineer has to do a review the code of its colleague Data Scientist for production environment, it is always important to understand where it is best to put the focus. Often, the best approach is to promote the resources awareness to be used and to find a framework to split the work together.
https://2024.incontrodevops.it/talks_speakers/
AWS User Group Padova event.
Which AWS services there are for forecasting scenarious ?
https://www.meetup.com/it-IT/aws-user-group-padova/events/298480058/
From your laptop to all resource that you need - 2023-12-09Alessandra Bilardi
PyData Impact Scholars - PyData Global 2023
Imagine you are processing your data and your ML system from your laptop and there are not enough resources, but by adding a few lines of code you can access all the resources you need. So, from your Jupyter notebook you can orchestrate tests of your code and then you can run the same code in the cloud with … a flag and little else.
https://github.com/bilardi/pydata-global/tree/master/2023
PyDataVE #13
Which open source libraries can compete with Pandas, PySpark with some activities as apply, groupby and sum ?
https://www.meetup.com/pydata-venice/events/296507635/
Automation: from local test to production deploy - 2020-11-05Alessandra Bilardi
CloudConf 2020 in Streaming
Talk about a sample of Automation Solution from local to production
https://2020.cloudconf.it/
https://github.com/bilardi/aws-saving/
https://github.com/bilardi/aws-simple-pipeline/
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
2. Alessandra Bilardi
Data & Automation Specialist
● AWS User Group Venezia member
● Coderdojo member
● PyData Venezia member
alessandra.bilardi@gmail.com
@abilardi
bilardi
3. Alessandra Bilardi
Data & Automation Specialist
● AWS User Group Venezia member
● Coderdojo member
● PyData Venezia member
alessandra.bilardi@gmail.com
@abilardi
bilardi
4. Alessandra Bilardi
Data & Automation Specialist
● AWS User Group Venezia member
● Coderdojo member
● PyData Venezia member
alessandra.bilardi@gmail.com
@abilardi
bilardi
5. Alessandra Bilardi
Data & Automation Specialist
● AWS User Group Venezia member
● Coderdojo member
● PyData Venezia member
alessandra.bilardi@gmail.com
@abilardi
bilardi
6. Alessandra Bilardi
Data & Automation Specialist
● AWS User Group Venezia member
● Coderdojo member
● PyData Venezia member
alessandra.bilardi@gmail.com
@abilardi
bilardi