Web-Scale Graph Analytics with Apache Spark with Tim HunterDatabricks
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems of understanding and information within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implements graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, we’ll discuss the work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations of connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; see its performance in the context of other popular graph libraries; and hear about real-world applications.
My talk at the Stockholm Natural Language Processing Meetup. I explained how word2vec is implemented and how to use it in Python with gensim. When words are represented as points in space, the spatial distance between words describes a similarity between these words. In this talk, I explore how to use this in practice and how to visualize the results (using t-SNE)
A MapReduce job usually splits the input data-set into independent chunks which are processed by the map tasks in a completely parallel manner. The framework sorts the outputs of the maps, which are then input to the reduce tasks. Typically both the input and the output of the job are stored in a file-system.
Web-Scale Graph Analytics with Apache Spark with Tim HunterDatabricks
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems of understanding and information within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implements graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, we’ll discuss the work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations of connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; see its performance in the context of other popular graph libraries; and hear about real-world applications.
My talk at the Stockholm Natural Language Processing Meetup. I explained how word2vec is implemented and how to use it in Python with gensim. When words are represented as points in space, the spatial distance between words describes a similarity between these words. In this talk, I explore how to use this in practice and how to visualize the results (using t-SNE)
A MapReduce job usually splits the input data-set into independent chunks which are processed by the map tasks in a completely parallel manner. The framework sorts the outputs of the maps, which are then input to the reduce tasks. Typically both the input and the output of the job are stored in a file-system.
Apache Sqoop: A Data Transfer Tool for HadoopCloudera, Inc.
Apache Sqoop is a tool designed for efficiently transferring bulk data between Hadoop and structured datastores such as relational databases. This slide deck aims at familiarizing the user with Sqoop and how to effectively use it in real deployments.
This is a deck of slides from a recent meetup of AWS Usergroup Greece, presented by Ioannis Konstantinou from the National Technical University of Athens.
The presentation gives an overview of the Map Reduce framework and a description of its open source implementation (Hadoop). Amazon's own Elastic Map Reduce (EMR) service is also mentioned. With the growing interest on Big Data this is a good introduction to the subject.
Lexical analyzer, tokenizer, scanner, or lexer is a function that is invoked by the syntax analyzer. This function returns the nxt lexicon or word in the source file.
Apache Sqoop: A Data Transfer Tool for HadoopCloudera, Inc.
Apache Sqoop is a tool designed for efficiently transferring bulk data between Hadoop and structured datastores such as relational databases. This slide deck aims at familiarizing the user with Sqoop and how to effectively use it in real deployments.
This is a deck of slides from a recent meetup of AWS Usergroup Greece, presented by Ioannis Konstantinou from the National Technical University of Athens.
The presentation gives an overview of the Map Reduce framework and a description of its open source implementation (Hadoop). Amazon's own Elastic Map Reduce (EMR) service is also mentioned. With the growing interest on Big Data this is a good introduction to the subject.
Lexical analyzer, tokenizer, scanner, or lexer is a function that is invoked by the syntax analyzer. This function returns the nxt lexicon or word in the source file.
Debugging IoT Sensor Interfaces (SPI) with Digilent Analog Discovery 2Vincent Claes
Tutorial on how to debug a SPI Peripheral with the Digilent Analog Discovery 2.
We create a SPI Master with the Discovery and use an Olimex MOD-LED8x8RGB Matrix as device under test.
Supercharging CI/CD with GitLab and Rancher - June 2017 Online MeetupShannon Williams
In our June Rancher meetup, GitLab joined us to discuss how to use GitLab tools and the Rancher container management platform for an automated, scalable CI pipeline. We were also joined by the devops team at Kloeckner.i who demonstrated their real-world example of the two together.
Freelancer Weapons of mass productivityGregg Coppen
In the battle to stay organized, efficient, sane and maximize on billable time it helps to have systems in place to help deal with the daily business processes and management that make sure that you are working on what you should be and that projects, budgets and timelines stay on track. In particular, when you work on your own, its critical to have things like billing, time tracking and project management as a natural and seamless part of your workflow.
This session aims to be a whistle stop tour of some useful open source tools and subscription solutions I have found to be well worth their costs - including how they can be used effectively together to allow you to make the most efficient use of your time designing and developing Drupal sites.
I work as a remote contractor & consultant and my clients are drupal shops and companies needing web sites and systems designed, built, themed and/or maintained. These tools and services work for me to help stay organized and on top of my workload and help me to manage my responsibilities across multiple clients and timezones effectively.
The material in this session is geared more towards individual freelancers although much of it will be relevant for larger drupal shops and teams too.
A few of the topics I intend to cover will include
* Project Management with Redmine - an overview of this powerful open source project management system and a demo of some of the plugins that extend its functionality and integrate well with Drupal, Dropbox, Github, Chrome and others.
* Simplifying getting paid and easy record keeping - Easy invoicing, credit card processing and automatic importing of expenses using Freshbooks & Stripe
* Design to theme tricks and up and coming in-browser design tools and workflows using Styletiles, CSS Hat, SASS, Typekit, Typecast & Livestyle
* Faster Drupal development tips using Alfred & Sublime Text
* Rapid protoyping using Bootstrap/Zenstrap
* Site building strategies using install profiles and drush make files
* Deployment and Maintenance using Aegir
* Server monitoring using New Relic & load testing using Blazemeter
* Hosting and managing your site in the cloud
It is my aim to introduce ( in some cases briefly) tools and services that have made a difference to me that may have the potential to add to and improve your existing workflows.
This is introduction to Git, distributed version control system. You will learn about git history, reasons behind its invention, design considerations, internal structure and see how to use git for your projects.
ZF2 takes a different approach to services; there are several services out there and you should be providing the ability for ZF2 to integrate with this. ZF2 marries services with composer and a different packaging mechanism to ensure that services can be released without a specific framework version. This not only helps the framework but helps you prevent an API changing in between framework releases without having an issue of awaiting a framework release.
From its humble beginning as a place where people would pay $5 to get a funny video, Fiverr has grown into the world’s largest marketplace for digital services.
Along the way, our frontend architecture has had to evolve as well - with technologies changing at a rapid pace and frontend developers in general always wanting to work with the latest, shiniest thing, not being adaptable to the environment around you can easily lead you down a road where your stack can’t support your needs & where you’re constantly playing catch-up to whatever it is everyone else is doing.
In this talk, I’ll give an overview of the FE path that Fiverr took — where we started, what we’re currently doing and where we’re (hopefully) going.
How to convert your Full Trust Solutions to the SharePoint Framework (SPFx)Brian Culver
This is a walkthrough on the tools, steps and process for converting common full trust solution to SharePoint Framework solutions. We will cover several scenariosn and discuss how to handle each appropriately. We have established a general workflow for converting your full trust solutions to SPFx solutions that I will share. We also will cover the proper configuration for your development environment. Lastly I will quickly highlight and show the process for building an SPFx solution and deploying it to Office 365. Lots of DO’s and DON’Ts will be shared. I’ll show you some of my scars too. From this session forward, you will want to hone your skills in modern SharePoint and convert everything to SPFx solutions.
Attendee Takeaways:
1. Understand the why, the how, and what make up the SharePoint Framework (SPFx).
2. A demonstration where we take a common Full Trust Solutions and covert it to the SharePoint Framework (SPFx).
3. I share lots of tips, DO’s and DON’Ts to save you hours and days of your life
Using Angular-CLI to Deploy an Angular 2 App Using Firebase in 30 MinutesTracy Lee
Do you think it’s possible to live code an Angular 2 app, create a backend, and deploy it in one talk? With the convenience of Angular-CLI and a little help from Firebase and material design, see how easy it is to create apps with Angular 2 and why this new framework is the great for quickly prototyping ideas and is built to scale large applications across distributed teams.
Example repo with completed branches - https://github.com/ladyleet/ngcruise-up
Deployed App - https://ngpoland-d7d54.firebaseapp.com/
Join us for a look behind the curtain at GitHub Enterprise's new features, following its 2.10 release. Dive into an intermediate level auto-deployment scenario using GitHub, Node, Grunt, and Jenkins along with AWS Technologies including the AWS Command Line Interface, Lambda, Amazon S3, and Identity and Access Management.
Speaker: Daniel Figucio is the Director of Solutions Engineering, ASEAN and ANZ @ GitHub
Level: 200
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)Brian Culver
This is a walkthrough where we convert a common full trust solution to a SharePoint Framework solution. I will show you the general workflow for converting your full trust solutions to SPFx solutions. I show you the proper configuration for your development environment. We walk through building the SPFx solutions and deploying it to Office 365. Lots of DO’s and DON’Ts will be shared. I’ll show you some of my scars too. From this session forward, you will want to hone your skills in modern SharePoint and convert everything to SPFx solutions.
Attendee Takeaways:
1. Understand the why, the how, and what make up the SharePoint Framework (SPFx).
2. A demonstration where we take a common Full Trust Solutions and covert it to the SharePoint Framework (SPFx) in less than 1 hour.
3. I share lots of tips, DO’s and DON’Ts to save you hours and days of your life. Yes, you are welcome :)
The AWS SDK for Go is now live and ready for developers to build productive applications with using the Go programming language. This session will show you how to get started using the AWS SDK for Go. We’ll introduce the best practices for how to use the SDK, and how it integrates with your Go application. We will explore an example application which uses async programming with Amazon S3 and Amazon SQS to distribute tasks across multiple concurrent goroutines, and use Amazon Elastic Beanstalk to deploy and scale the application.
Similar to R Markdown, Rpubs & github publishing and Shiny by Example (20)
Step by Step tutorial on the implementation of FreeRTOS on AVNET MiniZED Board. This board is powered by a Xilinx Zynq FPGA (7007S).
This manual uses Xilinx Vitis Environment.
Implementing an interface in r to communicate with programmable fabric in a x...Vincent Claes
This paper shows the details for implementing an interface between the programming language R and programmable fabric of a Xilinx Zynq FPGA on a zedboard.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
36. • In Rstudio make a new directory “gh-pages”
• Move your files into this directory
Online publishing to github
Vincent Claes
37. • In Rstudio make a new branch “gh-pages”
• Push /commit the files to this new branch and check them on
github.com website
• Knit to html your Example02.Rmd file
Online publishing to github
Vincent Claes
38. • Knit Rmd to html and push / commit
Online publishing to github
Vincent Claes
39. • When your url is: https://github.com/pxltech/SmartSystems/tree/gh-
pages/gh-pages change it to
https://pxltech.github.io/SmartSystems/gh-pages/Example02.html
Online publishing to github
Vincent Claes
40. • Generating slides
• Generating pdf files
• Generating WORD documents
Rmarkdown: some other features
Vincent Claes
41. • File => new project => New Directory => Shiny Web Application
Shiny Applications in R
Vincent Claes
42. • Create git repository
Shiny Applications in R
Vincent Claes
46. Shiny Applications in R Publish application
Vincent Claes
• http://www.shinyapps.io create a free account
• Publish your application to shinyapps.io website