This document discusses accessing and preparing data for analysis in WebFOCUS. It covers getting data by uploading files from a PC or accessing server data sources. It also covers preparing data through profiling to understand it and deriving new variables. The document demonstrates sampling to improve performance on large datasets and combining data through union or join operations. It provides examples on the Citi Bike NYC bike sharing dataset to demonstrate preparing and profiling the data to answer different analysis questions.

1. Accessing and Preparing Data
Summit 2020
Clif Kranish

2. Contents
2
Get Data
Use a representative sample of your data source
Sampling
Upload data from your PC or access server data sources
Understand your data and make it more useful
Profiling and Preparing Data
1
2
3
4
5
Augment your data from other data sources
Combining Data
Stage your data for analysis
Loading Data

3. “Data preparation is the act of manipulating
raw data into a form that can be readily and
accurately analyzed.”
3
— Wikipedia

4. Upload data from your desktop or
access tables and files on the server.
Get Data
4

5. Preparing Data
5
WebFOCUS is used with data from disparate sources
Some data sources can be used as-is
Some may be need to prepared prior to use
• Company star schema data warehouses
• Departmental data marts
• Other curated corporate data sources
• Personal database tables
• Publicly accessible data
• Excel worksheets and other files on a PC

6. Get Data
• Upload Local Files
• Connect to Server Files and
databases

7. Prepare Data

8. Citi Bike (NYC Bike Share System)

9. Can we answer these questions from the data?
● How long is the typical ride?
● What hours of the day has the most rides?
● Has the number of rider per day varied over the month?
● What generation are the riders?
● What neighborhoods do the most rides start from?

10. Yes, if we augment the data with additional information
● Trip duration in minutes
● Trip start time as hour of the day
● Trip day of the month
● Station zip code, neighborhood, city and county
● Generation and age in years

11. 11
Get Data
Local Files or Server Files

12. Upload Data
Data is stage (uploaded) and previewed as Raw or Formatted

13. Upload data (delimited file)
Raw data shows original file

14. Upload Data
Select Excel workbook with multiple worksheets

15. Upload Data
Select workbook with multiple worksheets

16. Upload Data (from Excel)
Rows to skip and use as headers automatically suggested

17. Upload Data (from delimited file)
Header row and delimiter automatically suggested

18. Upload Data
Adapter and Application Folder default – or can be selected

19. Upload Data
Metadata and table names suggested

21. Upload Data
Choose columns to upload

22. Upload Data
Load

23. Server Files
Add new adapter

24. Sampling
24

25. Sampling
Use a Representative Sample instead of entire dataset (or first nnn rows)
• Improves responsiveness when preparing large data sets
• Automatically calculates sample size
• Confidence Level 99% - Margin of Error +/- 1%
• Stages a random sample
on disk… or in a database table

26. New Flow

27. Sampling
Enable Sampling

28. Sampling
Always enable sampling
Tools > Workspace > Settings > Settings for Web Console Preferences
Data Assist (Representative Sampling) > ENABLE_SAMPLING

29. Sampling
Hover to see sample size

30. Sampling
Recreate (Stratified) Sampling

31. Stratified Sampling
Ensure at least one row for every unique value of selected column(s)

32. Profiling and Preparing Data
32
Data Profiling helps to understand data
Preparing data can make data easier to analyze

33. Profile Data
See how data values are distributed

34. Prepare Data
Derive Trip Duration in Minutes

35. Prepare Data
Trip Duration in Minutes

36. Prepare Data
Derive Day of Month

37. Prepare Data
Derived day of month

38. Profile Data
Day of Month

39. Prepare Data
Derive Hour of Day

40. Prepare Data
Derive Hour of Day – Use function DTIME

41. Prepare Data
Derived Hour of Day – Sorted by Value

42. Profile Data – Birth Year
A cyclist was born in 1896? And what’s up with 1969?

43. Prepare Data
Birth Year - Select 1969 – Brushing shows values for other columns

44. Prepare Data
Birth Year – Add new expression

45. Prepare Data
Birth Year – Valid or NULL

46. Prepare Data
Birth Year – Valid or NULL

47. Prepare Data
Generations – Create Groups

48. Prepare Data
Generations – Create Groups

49. Prepare Data
Generations – Create Group - Millennials

50. Combine Data
50
Augment data with additional information from other
files with UNION or JOIN

51. Combine Data Sources
Select JOIN or UNION

52. UNION
Multiple data sources with the same layout

53. JOIN
Two tables with a column with shared values

54. Join Editor
Columns to join are suggested. See effects of inner or outer join

55. Profile
Zip Code, Neighborhood, City, County for each starting station

56. Stage data for analysis in a database
table, column store or as a view
Load
56

57. Target
New Target

58. New Target
Load Options

59. Load Options
Select Adapter, Connection, Synonym and Table

60. Run
Or Schedule or Save

61. Thank you!
61