This document outlines the content of a five-day course on advanced interactive visualization using the Shiny framework in R. Day 1 introduces visual analytics. Days 2-3 cover visualization methods for unstructured and structured data. Day 4 focuses on advanced interactive visualization. Day 5 teaches how to build Shiny apps for interactive data exploration and presentation. The document provides instructions on downloading required materials and libraries to follow along with examples and practice building a basic Shiny app using UI, server logic, and reactive programming.
Logical Inference in a Hyper-Relational DatabaseVaticle
Inference is something we humans do all the time. Given a set of facts about the world, we derive new ones using some form of inference. Automated reasoning has been studied extensively but its value in providing a more powerful abstraction layer for database languages has been overlooked so far.
This talk explores deductive inference in Grakn, a hyper-relational database that has automated inference as one of its core features. Rather than defining SQL views or writing ad hoc code, in Grakn we can define logical rules that provide a more intuitive way to describe higher level domain concepts. In the talk we give a quick overview of computational logic semantics and of top-down and bottom-up inference algorithms. Then, after introducing some preliminary Grakn concepts, we show how logical rules are resolved in a query.
Introduction to Knowledge Graphs with Grakn and Graql Vaticle
Cognitive/AI systems process knowledge that is far too complex for current databases. They require an expressive data model and an intelligent query language to perform knowledge engineering over complex datasets.
In this talk, we will discuss how Grakn, a database to organise complex networks of data and make it queryable, provides the knowledge graph foundation for intelligent systems to manage complex data.
We will discuss how Graql, Grakn's reasoning (through OLTP) and analytics (through OLAP) query language, provides the tools required to do the job: a knowledge schema, a logical inference language, a distributed analytics framework.
And finally, we will discuss how Graql’s language serves as unified data representation of data for cognitive systems.
Data Visualization: Introduction to Shiny Web ApplicationsOlga Scrivner
In this workshop, I will introduce you to the concept of Declarative Reactive Web Frameworks, allowing for interactive user-friendly data visualization and data analytics, particularly Shiny. Shiny is an R package that creates interactive applications for data visualization. You will learn some Shiny basics: how to build your reactive app and deploy it to the server
Logical Inference in a Hyper-Relational DatabaseVaticle
Inference is something we humans do all the time. Given a set of facts about the world, we derive new ones using some form of inference. Automated reasoning has been studied extensively but its value in providing a more powerful abstraction layer for database languages has been overlooked so far.
This talk explores deductive inference in Grakn, a hyper-relational database that has automated inference as one of its core features. Rather than defining SQL views or writing ad hoc code, in Grakn we can define logical rules that provide a more intuitive way to describe higher level domain concepts. In the talk we give a quick overview of computational logic semantics and of top-down and bottom-up inference algorithms. Then, after introducing some preliminary Grakn concepts, we show how logical rules are resolved in a query.
Introduction to Knowledge Graphs with Grakn and Graql Vaticle
Cognitive/AI systems process knowledge that is far too complex for current databases. They require an expressive data model and an intelligent query language to perform knowledge engineering over complex datasets.
In this talk, we will discuss how Grakn, a database to organise complex networks of data and make it queryable, provides the knowledge graph foundation for intelligent systems to manage complex data.
We will discuss how Graql, Grakn's reasoning (through OLTP) and analytics (through OLAP) query language, provides the tools required to do the job: a knowledge schema, a logical inference language, a distributed analytics framework.
And finally, we will discuss how Graql’s language serves as unified data representation of data for cognitive systems.
Data Visualization: Introduction to Shiny Web ApplicationsOlga Scrivner
In this workshop, I will introduce you to the concept of Declarative Reactive Web Frameworks, allowing for interactive user-friendly data visualization and data analytics, particularly Shiny. Shiny is an R package that creates interactive applications for data visualization. You will learn some Shiny basics: how to build your reactive app and deploy it to the server
Jenkins NS delivered a session on Consume Graph APIs in SPFx
Agenda:
• Overview of MS Graph
• Office 365 APIs
• What is Graph API?
• Consume Microsoft Graph
• AadHttpClient object
• MSGraphClient object
• API permissions requests
• Isolated web parts
Olga Mierzwa-Sulima talk from eRum 2018 conference.
Talks cover 6 new Shiny packages that Appsilon has developed to extend Shiny's frontend possibilities:
1. Shiny Sementic - Package for SemanticUI components
2. semantic.dashboard - Package for SemanticUI Shiny dashboard
3. shiny.router - Making routing possible with Shiny
4. shiny.i18n - Helps build multi language apps
5. shiny.users - Authentication handled by Shiny done on the application side
6. shiny.admin - Allowing to collect statistics about users activity in the App
blogpost: https://appsilon.com/why-you-should-regret-not-going-to-erum-2018/
github: https://github.com/Appsilon
Youtube: https://youtu.be/6hISTA8fGiA
Presentation is highlighting novelties in SPA development with Angular 2 (+Ionic 2 demo) with real code examples.
We created together simple Ng2 application with Angular CLI.
All the code is available on GitHub (link to demos is at the end of presentation).
Prerequisites:
1. Install NodeJS. It is better to install version 6 or 4x. Read about NPM.
2. Install TypeScript + editor (Visual Studio Code or Sublime 3).
3. Install Angular 2 Command Line Interface (Angular CLI):
npm install -g angular-cli
These slides focus on documentation for REST APIs. See http://idratherbewriting.com for more detail. For the video recording, see http://youtu.be/0yfNd7tzH2Q. This deep dive is the second slide deck I used in the presentation.
This presentation is dedicated to studying the fundamentals of Angular 2.
To follow along with the presentation, watch this 3-part YouTube Series here: http://bit.ly/2mnLZNz
You can also download Traversy's Spotify App here: http://bit.ly/2m1TxI3
Introduction to Interactive Shiny Web ApplicationOlga Scrivner
2 hour hands-on workshop on how to create, deploy and use Shiny in research and teaching. The materials for the workshop are https://languagevariationsuite.wordpress.com/2018/11/27/introduction-to-interactive-shiny-web-applications
Tech leaders guide to effective building of machine learning productsGianmario Spacagna
Part 2/2 (Tech Leaders)
Data and Machine Learning (ML) technologies are now widespread and adopted by literally all industries. Although recent advancements in the field have reached an unthinkable level of maturity, many organizations still struggle with turning these advances into tangible profits. Unfortunately, many ML projects get stuck in a proof-of-concept stage without ever reaching customers and generating revenue. In order to effectively adopt ML technologies, enterprises need to build the right business cases as well as to be ready to face the inevitable challenges. In this talk, we will share common pitfalls, lessons learned, and best practices, while building different enterprise products. In particular, we will focus on the generic use case of ML as the core technology enabling customer-facing products regardless of the specific industry or application.
You will:
Understand if ML is the right solution for your business and set the right expectations;
Deal with the additional uncertainty of ML projects with respect to traditional software;
Build a balanced ML team and cover the broad spectrum of skills;
Know how to apply the scientific workflow in an agile development framework;
Learn how to turn research into production systems including engineering practices and tools;
Be able to leverage modern cloud and serverless architecture for scalable, autonomous and cheaper deployments.
Angular 2 has finally hit the shelves and it is not just an upgrade. The producers of Angular have issued Angular 2 and it stands miles apart from the original framework. The new Angular 2 is a modern and robust framework that is faster, more expressive and flexible in nature. Here are a few interesting facts about Angular 2 that you may need to get started with this brilliant framework.
Jenkins NS delivered a session on Consume Graph APIs in SPFx
Agenda:
• Overview of MS Graph
• Office 365 APIs
• What is Graph API?
• Consume Microsoft Graph
• AadHttpClient object
• MSGraphClient object
• API permissions requests
• Isolated web parts
Olga Mierzwa-Sulima talk from eRum 2018 conference.
Talks cover 6 new Shiny packages that Appsilon has developed to extend Shiny's frontend possibilities:
1. Shiny Sementic - Package for SemanticUI components
2. semantic.dashboard - Package for SemanticUI Shiny dashboard
3. shiny.router - Making routing possible with Shiny
4. shiny.i18n - Helps build multi language apps
5. shiny.users - Authentication handled by Shiny done on the application side
6. shiny.admin - Allowing to collect statistics about users activity in the App
blogpost: https://appsilon.com/why-you-should-regret-not-going-to-erum-2018/
github: https://github.com/Appsilon
Youtube: https://youtu.be/6hISTA8fGiA
Presentation is highlighting novelties in SPA development with Angular 2 (+Ionic 2 demo) with real code examples.
We created together simple Ng2 application with Angular CLI.
All the code is available on GitHub (link to demos is at the end of presentation).
Prerequisites:
1. Install NodeJS. It is better to install version 6 or 4x. Read about NPM.
2. Install TypeScript + editor (Visual Studio Code or Sublime 3).
3. Install Angular 2 Command Line Interface (Angular CLI):
npm install -g angular-cli
These slides focus on documentation for REST APIs. See http://idratherbewriting.com for more detail. For the video recording, see http://youtu.be/0yfNd7tzH2Q. This deep dive is the second slide deck I used in the presentation.
This presentation is dedicated to studying the fundamentals of Angular 2.
To follow along with the presentation, watch this 3-part YouTube Series here: http://bit.ly/2mnLZNz
You can also download Traversy's Spotify App here: http://bit.ly/2m1TxI3
Introduction to Interactive Shiny Web ApplicationOlga Scrivner
2 hour hands-on workshop on how to create, deploy and use Shiny in research and teaching. The materials for the workshop are https://languagevariationsuite.wordpress.com/2018/11/27/introduction-to-interactive-shiny-web-applications
Tech leaders guide to effective building of machine learning productsGianmario Spacagna
Part 2/2 (Tech Leaders)
Data and Machine Learning (ML) technologies are now widespread and adopted by literally all industries. Although recent advancements in the field have reached an unthinkable level of maturity, many organizations still struggle with turning these advances into tangible profits. Unfortunately, many ML projects get stuck in a proof-of-concept stage without ever reaching customers and generating revenue. In order to effectively adopt ML technologies, enterprises need to build the right business cases as well as to be ready to face the inevitable challenges. In this talk, we will share common pitfalls, lessons learned, and best practices, while building different enterprise products. In particular, we will focus on the generic use case of ML as the core technology enabling customer-facing products regardless of the specific industry or application.
You will:
Understand if ML is the right solution for your business and set the right expectations;
Deal with the additional uncertainty of ML projects with respect to traditional software;
Build a balanced ML team and cover the broad spectrum of skills;
Know how to apply the scientific workflow in an agile development framework;
Learn how to turn research into production systems including engineering practices and tools;
Be able to leverage modern cloud and serverless architecture for scalable, autonomous and cheaper deployments.
Angular 2 has finally hit the shelves and it is not just an upgrade. The producers of Angular have issued Angular 2 and it stands miles apart from the original framework. The new Angular 2 is a modern and robust framework that is faster, more expressive and flexible in nature. Here are a few interesting facts about Angular 2 that you may need to get started with this brilliant framework.
Introduction to Web Scraping with PythonOlga Scrivner
In this workshop, you will learn how to extract web data with Beautiful Soup, a Python library for extracting data out of HTML- and XML-structured documents. You will also learn the basics of scraping and parsing data. In this hands-on workshop, we will also be using the DataCamp platform and participants are requested to have a free account with DataCamp prior the workshop.
Call for paper Collaboration Systems and TechnologyOlga Scrivner
Our minitrack encourages research contributions that deal with learning theories, cognition, tools and their development, enabling platforms, communication media, distance learning, supporting infrastructures, user experiences, research methods, social impacts, learning analytics, and measurable outcomes as they relate to the area of technology and its support of improving teaching and learning. In particular, the significant increase of online and distributed classroom environments brings new technological challenges.
Hands-on workshop on Jupiter Notebook and Machine Learning.
The link to material - https://languagevariationsuite.wordpress.com/2020/02/22/machine-learning-101-with-jupyter-notebook/amp/
CEWIT Hand-on workshop.
Link to materials - https://languagevariationsuite.wordpress.com/2020/01/31/faculty-accelerator-crash-course-rmarkdown-with-r-introduction/amp/
Video of Workshop - https://media.dlib.indiana.edu/media_objects/rj430941s
This is workshop offered via Social Science Research Center to students and faculty to become familiar with an online collaborative writing using Latex and Overleaf.
Workshop nwav 47 - LVS - Tool for Quantitative Data AnalysisOlga Scrivner
In the format of hands-on session, this workshop will introduce participants to the Language Variation Suite (LVS), a user-friendly interactive web application built in R. LVS provides access to advanced statistical methods and visualization techniques, such as mixed-effects modeling, conditional and random tree analyses, cluster analysis. These advanced methods enable researchers to handle imbalanced data, measure individual and group variation, estimate significance, and rank variables according to their significance.
Gender Disparity in Employment and EducationOlga Scrivner
Data analysis is presented at IndyBigData Visualization Challenge 2018. Data is provided by MPH - see https://www.indybigdata.com/visualization-challenge/
CrashCourse: Python with DataCamp and Jupyter for BeginnersOlga Scrivner
Crash course for beginners is based on Python Introduction by Philip Schowenaars from DataCamp and Jupyter Introduction adapted from Adapted from Pryke, B. (2018). Jupyter Notebook for Beginners: A Tutorial. DataQuest. https://www.dataquest.io/blog/jupyter-notebook-tutorial/
Optimizing Data Analysis: Web application with ShinyOlga Scrivner
In the format of hands-on session, this workshop will introduce participants to the Language Variation Suite (LVS), a user-friendly interactive web application built in R. LVS provides access to advanced statistical methods and visualization techniques, such as mixed-effects modeling, conditional and random tree analyses, cluster analysis. These advanced methods enable researchers to handle imbalanced data, measure individual and group variation, estimate significance, and rank variables according to their significance.
Workshop files:
Categorical data csv – Use of R in New York (Labov 1966) - http://cl.indiana.edu/~obscrivn/docs/categoricaldata.csv
Continuous data csv – Intervocalic /d/ (Díaz-Campos et al. 2016) - http://cl.indiana.edu/~obscrivn/docs/continuousdata.csv
Language Variation Suite - https://languagevariationsuite.shinyapps.io/Pages/
Data Analysis and Visualization: R WorkflowOlga Scrivner
The lecture introduces to R project set-up, planning and deploying as well as to the concept of tidy data (Wickham and Grolemund, 2017).
Visual Insights Talks 2018 at
http://ivmooc.cns.iu.edu/
http://cns.iu.edu/
Quantitative Data AnalysisReliability Analysis (Cronbach Alpha) Common Method...2023240532
Quantitative data Analysis
Overview
Reliability Analysis (Cronbach Alpha)
Common Method Bias (Harman Single Factor Test)
Frequency Analysis (Demographic)
Descriptive Analysis
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
2. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
What You Will Learn
DAY 1 Introduction to Visual Analytics
DAY 2 Visualization Methods, Design, and Tools
DAY 3 Working with Unstructured Data
DAY 4 Working with Structured Data
DAY 5 Advanced Interactive Visualization with
Shiny
6. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Quick Overview: Visual Analytics and Linguistics
“ Language is a system of relatively arbitrary symbols”
(Baker-Shenk and Cokely, 1980)
http://www.tableaufit.com/tableau-conference-linguistics-data-visualization/
8. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Visualization Tools
Challenges:
Many tools are built for non-linguistic research
Limited preprocessing
Visualization may not fit linguistic data
9. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Visualization Tools
Challenges:
Many tools are built for non-linguistic research
Limited preprocessing
Visualization may not fit linguistic data
Solution: Reactive Framework
Interactive and web-based
Research-driven customization
Build your own tool!
10. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Reactive Web Framework
“Reactive Systems are highly responsive, giving users
effective interactive feedback”
http://www.reactivemanifesto.org/
http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/
12. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Reactive Framework and Data Science
“The impact of data scientists’ work depends on how well
others can understand their insights to take further actions”
Benefit 1: Interactive display and manipulation of data
Benefit 2: No installation required
Benefit 3: Easy to develop and share with clients and
project teams
Benefit 4: Open source library
http://datascience.ibm.com/blog/shiny-a-data-scientist-best-friend/
13. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Shiny Application
1. Shiny is an R package for building interactive web
applications
2. Open-Sourced by RStudio 11/2012 on CRAN
3. Uses web sockets (new HTTP):
Interactive communication sessions between the user’s
browser and a server without having to poll the server
for a reply
4. Entirely extensible - custom input/output
15. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Shiny Gallery - Get Inspired - 5 min
https://www.rstudio.com/products/shiny/
shiny-user-showcase/
Shiny apps for teaching
Shiny apps for analytics
16. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Show me Shiny - Get Inspired - 5 min
https:
//www.rstudio.com/products/shiny/
shiny-user-showcase/
Select one or two applications, examine the app and the
code
17. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Workshop Materials
1. Rstudio
2. Install and activate shiny library
3. Materials:
Zip file - shinyworkshopfiles
https:
//github.com/Jivitesh-Poojary/CNS-Shiny-Apps
37. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
New Input in UI.R
Let’s look at the fileInput() function from Shiny
Reference page
Type in your browser: fileInput Shiny
or go to the Shiny Reference page -
https://shiny.rstudio.com/reference/shiny/
latest/fileInput.html
40. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Reactive Function in Sever.R
Reactive - it changes every time the user uploads new data
Create a function that reads csv file
myfile() is a function that will read and return csv files
48. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
One more IF-Statement - Server.R
We want to do a histogram for csv file:
if (condition) {do..} else {do...}
output$distPlot <- renderPlot({
if (is.null(input$file1)) {
....
hist(x, breaks = bins, col = ’darkgray’, border =
’white’)
}
49. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
One more IF-Statement - Server.R
We want to do a histogram for csv file:
if (condition) {do..} else {do...}
output$distPlot <- renderPlot({
if (is.null(input$file1)) {
....
hist(x, breaks = bins, col = ’darkgray’, border =
’white’)
}
else{
x <- myfile()$budget
bins <- seq(min(x), max(x), length.out = input$bins
+ 1)
hist(x, breaks = bins, col=’red’,
main = ’My New Histogram’)
}
})
57. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Future of Visualization Tools: CNS
Cyberinfrastructure for Network Science Center at Indiana
University - http://cns.iu.edu/
Our current project on Shiny Framework (team Olga
Scrivner and Jivitesh Poojary)
Build Shiny templates for Data Visualization: stage I
https:
//github.com/Jivitesh-Poojary/CNS-Shiny-Apps
Create user-friendly customizable Shiny dashboards:
stage II
Develop preprocessing plugins for Shiny templates:
stage III
58. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Create Your Own Shiny
Select one of the application from Zip files:
Word cloud with the library wordcloud2
Network graph with the library igraph
Streamgraph with the library streamgraph
Streamgraph is not available through install
You will need to install first the library devtools
Run:
library(devtools)
devtools::install github("hrbrmstr/streamgraph")
library(streamgraph)
59. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Useful resources
1. Shiny official tutorial -
http://shiny.rstudio.com/tutorial
2. Cheat sheet - http://shiny.rstudio.com/images/
shiny-cheatsheet.pdf
3. Publish your app free - http://www.shinyapps.io
4. Examples -http://www.showmeshiny.com/
5. Tutorial by Dean Attali - http://deanattali.com/
blog/building-shiny-apps-tutorial/
61. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
References
B¨orner, Katy. 2015. Atlas of Knowledge: Anyone Can Map. The
MIT Press
Collins, Christopher. 2012. Bridging the Linguistic Visualization
Divide. LingVis/UNCLH Workshop
Grolemund, Garrett. 2017. How to start with Shiny, Part 1.
Workshop
62. Visualization
Analytics:
Shiny Framework
Olga Scrivner
Course Info
Reactive
Framework
Shiny App
Practice
Credits
http:
//deanattali.com/blog/building-shiny-apps-tutorial/
http://scimaps.org/mapdetail/stream_of_scientific_128
https://github.com/IBMDataScience/dsx-shiny-apps
http://www.slideshare.net/SarahAerni/
data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-
http://www.unixstickers.com/image/data/stickers/
react/badge/React-JS.sh.png
https://github.com/rstudio/shiny/issues/250
http://www.slideshare.net/ilio-catallo/
spring-mvc-the-basics