デブサミ2017【17-B-5】の講演資料です。 http://event.shoeisha.jp/devsumi/20170216/session/1291/

2. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

3. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

4. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

5. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

6. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

7. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

8. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

9. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

10. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

11. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

12. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

13. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

14. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

15. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

16. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

17. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

18. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

19. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

20. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

21. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

22. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

23. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

24. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

25. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

26. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

27. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

28. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

29. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

30. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
• ( https://ring.education/ )
-
- https://ring.education/research/2016_hs_al_experiment_report_02/

31. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

32. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

33. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

34. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

35. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

36. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

37. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

38. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

39. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

40. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

41. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
id grade_id gender school_id
AAA 001 female 0000X
BBB 002 male 0000Y
CCC 005 male 0000Z
id label
001
002
003
id label
0000X X
0000Y Y
0000Z Z
id grade gender school
AAA female X
BBB male Y
CCC male Z

42. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

43. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

44. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

45. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

46. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

47. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

48. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
Rails
Kinesis
Lambda
TD SDK
web JS app native
web Single Page Application

49. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

50. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
/(^o^)

51. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

52. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

53. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

54. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

55. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
embulk-input-mongodb
by @hakobera
embulk-input-postgresql
embulk-parser-json
embulk-filter-expand_json
embulk-filter-hash
by @kamatama41
embulk-filter-mask
by @beniyama
embulk-output-td

56. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

57. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

58. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

59. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
Embulk Hive / Presto

60. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

61. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
class DailyVideoRatingReport(luigi_td.Query):
type = 'hive'
database = LuigiConfig().main_db
report_database = LuigiConfig().report_db
source = ‘queries/hive/daily/history/daily_video_rating_report.sql’
jst_date = luigi.DateParameter(default=DateTimeUtil.get_jst_date())
def requires(self):
return [
DailyUserMasterCreate(self.jst_date),
DailyContentsMasterCreate(self.jst_date)
]

62. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
def run(self):
TdUtil.drop_partition(
table=‘video_rating_report’,
to_jst_date=self.jst_date
)
super(DailyVideoRatingReport, self).run()
def output(self):
return luigi_td.ResultTarget(
'{0}/DailyVideoRatingReport-{1}.txt'.format(
LuigiConfig().output_path,
self.jst_date.strftime('%Y-%m-%d')
),
'td://@/reports/video_rating_report?mode=append'
)

63. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

64. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
+video_rating_report:
+partial_delete:
td_partial_delete>: video_rating_report
database: reports
from: ${moment(session_time).utc().add("days", -1).unix()}
to: ${moment(session_time).utc().unix()}
+insert_records:
td>: queries/reporting/video_rating_report_insert.sql
database: reports
engine: presto
insert_into: video_rating_report

65. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
+fetch_records:
td>: queries/reporting/video_ratings_fetch.sql
database: reports
engine: presto
store_last_results: true
+slack_notification:
http>: https://hooks.slack.com/services/XXXXXXXX
method: POST
content:
text: “Average rating is ${td.last_results.average_ratings}”
link_names: true
content_format: json

66. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

67. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

68. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

69. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

70. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

71. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.
web console

72. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

73. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

74. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

75. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

76. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

77. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

78. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

79. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

80. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

81. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

82. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

83. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.

84. (C) Recruit Marketing Partners Co.,Ltd. All rights reserved.