Explained how big data analytics can apply for log files and what kind of user interface is appropriate for each case.

1. Big data Analytics On Log Files And UI Design Page 1
BigData Analytics On Log File And UI Design
Introduction:...............................................................................................................................1
Specific Focus:.............................................................................................................................1
Generic Focus:............................................................................................................................1
UI Design:...................................................................................................................................2
Introduction:
In belowsectionbrieflyexplainedhow bigdataanalyticscan applyfor log files and what kind of
user interface is appropriate for each case.
Specific Focus:
1. Identify security vulnerabilities
a. Anomaly Detection: Use k-means clustering algorithm to detect anomalies, outliers,
exceptions, malwares and so forth.
b. Identify DoSattacks(simulatemoreusers than websitecan handle): Normally result in an abnormal
number of requests (hits) in a short period of time and create firewall rules to block a
specific IP Address to avoid DoS.
2. Host/Date based event statistics
a. The process for returning all the events for a particular date.
b. All eventsinthe collectionforaparticularhost ona particulardate.Thiskind of analysis
may be useful for investigating suspicious behavior by a specific user.
3. Predictive analysis
a. Categorizing the errors, events and co relating the errors would provide predictive
analysis on system down.
4. Automated route cause analysis
a. E.g.: Whythe memoryutilizationishighatspecifictime?Correlate system log with web
logwithspecifictime andprovide the memoryutilizationishighbecause of highvolume
of user logged in as RCA.
Generic Focus:
Web Logs:
5. Visitors activity statistics:
a. Monthly, Weekly, Daily, Hourly visitors,
b. Monthly, Weekly, Daily, Hourly Hits,
c. Monthly, Weekly, Daily, Hourly Bandwidth,
d. Monthly, Weekly, Daily, Hourly Visit Duration
e. Monthly, Weekly, Daily, Hourly Page Views per Visitor
f. Monthly, Weekly, Daily, Hourly Pages by View Time
g. How long they staying in.

2. Big data Analytics On Log Files And UI Design Page 2
6. Visitors system statistics
a. Operating Systems used,
b. Browsers used,
7. Geo location statistics
a. N/W traffic by county/countries
b. Visitors
c. Hits
d. Bandwidth
e. Time served
8. Error statistics
SystemLogs (iostat,vmstat,netstat):
9. Predict future resource needs based on long-term predictive reports
a. These reports generate a long-term trend line for performance. The data from these
reportsisoften used with a linear regression to predict when additional RAMmemory
or CPU power is required for the server.
10. Capacity planning predictions
a. Use graph algorithmstopredicttoevaluate hardware resourcesrequiredforthe specific
environment.
UI Design:
1. Monthlycalendarheatmap,
 Bandwidth,Hits,Visitors countspermonthincalendarview.
2. Sparkline charts orderbymost loggedinforthat week.

3. Big data Analytics On Log Files And UI Design Page 3
 Col 1 : User id
 Col 2 : No.Of loginsforthatweek
 Col 3 : Monthlyloginsforthe usergroup by 4 weeks
 Col 4 : No.Of page viewsforthatweek
 Col 5 : Monthly page view forthe user groupby 4 weeks
 Col 6 : No.Of errorcodesreturnedforthat week
 Col 7 : Monthlyerror codesview forthe usergroup by4 weeks
3. Dailyactivities table andchart
 Drilldownlinksforeachhits,page views,visitorscolumns.
 Drilldownisuserbasedforthatday.

4. Big data Analytics On Log Files And UI Design Page 4
4. Popularpagesintable andchart format
 Drilldownlinksforeachhits,visitorscolumns.
5. No of hitsdistributedbymodule
 Claims,Provider,Member,Operationsandsoon

5. Big data Analytics On Log Files And UI Design Page 5
6. Infographictype display

6. Big data Analytics On Log Files And UI Design Page 6