Project MeniscusSimplex Internet / Research Institute / Jeong, Ki-Chul2013. 4. 29
Meniscus?• LAAS(Logging-as-a-Service) Backend API• Python based system• Event - collection, transit, processing• Large-sca...
SEE THE ANIMATED OVERVIEW
APIS
Tenant API• Tenant = {Host, Event Producer, Profile}• A Tenant = A Consumer• Event Producer = APP or Service• Profile = [E...
Tenant API• Requests to coordinator worker• Tenant, Host, Event Producer, Profile=> CRUD Available• has a TOKEN=> validate...
Correlator API• RequestsTo a Correlator worker serverFrom a Consumer servervia http(s)with a Message token(Tenant token)• ...
Coordinator API• Pairing- registers a worker• Configuration– requests a worker’s configuration• Status– updates a workers ...
Status API• Worker Status API (PUT, Manually)- System Load Average- Disk Usage• Coordinator Status API- grid status- indiv...
Broadcaster API• What’s Brodcaster API?I don’t know well!• Communicates with coordinator workerserver.• New Feature!
GETTING STARTED
Server deployment exampleHost1• Coordinator• MongoDBHost2• CorrelatorHost3• Storage
HOST1• Creating a development environment[LINK]• Install MongoDB[LINK]
uWSGI.ini
meniscus/etc/meniscus/meniscus.conf
Run uwsgi daemon!!# uwsgi uwsgi.ini
HOST2• Creating a development environment[LINK]• Run uwsgi daemon!!Default Persona : pairing
CURL Request• Attempts pairing with coordinator• Becomes correlation worker after automatically restart
HOST3• Creating a development environment[LINK]• Run uwsgi daemon!!Default Persona : pairing
CURL Request• Attempts pairing with coordinator• Becomes store worker after automatically restart
COMPLETED FOR SETTING UP
Logging a data• Already MUST register a tenant• Request to correlator server(host2)
FlowCorrelator(host2)Storage(host3)MongoDB(host1)
Conclusion• Well-made backend APIs for logging(Good architecture)• Distributed workers processing• Diverse Persona(worker)...
Not yet• Data Store – Only MongoDBHadoop / HDFS / HbaseElastic Search• Perfect documentationneed more examplesneed exact e...
The thing we can now do with meniscus isJUST to create log data into mongodb viarestful api and to fetch log data frommong...
THANKS
Upcoming SlideShare
Loading in …5
×

Project meniscus

360 views

Published on

a slide researched Project meniscus

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
360
On SlideShare
0
From Embeds
0
Number of Embeds
30
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Project meniscus

  1. 1. Project MeniscusSimplex Internet / Research Institute / Jeong, Ki-Chul2013. 4. 29
  2. 2. Meniscus?• LAAS(Logging-as-a-Service) Backend API• Python based system• Event - collection, transit, processing• Large-scale• Cloud Logging
  3. 3. SEE THE ANIMATED OVERVIEW
  4. 4. APIS
  5. 5. Tenant API• Tenant = {Host, Event Producer, Profile}• A Tenant = A Consumer• Event Producer = APP or Service• Profile = [Event Producer, …]• Host = Node , link to a profile(1:1)
  6. 6. Tenant API• Requests to coordinator worker• Tenant, Host, Event Producer, Profile=> CRUD Available• has a TOKEN=> validate, reset a token
  7. 7. Correlator API• RequestsTo a Correlator worker serverFrom a Consumer servervia http(s)with a Message token(Tenant token)• Stores a collection named “logs” in Mongo
  8. 8. Coordinator API• Pairing- registers a worker• Configuration– requests a worker’s configuration• Status– updates a workers status
  9. 9. Status API• Worker Status API (PUT, Manually)- System Load Average- Disk Usage• Coordinator Status API- grid status- individual worker status
  10. 10. Broadcaster API• What’s Brodcaster API?I don’t know well!• Communicates with coordinator workerserver.• New Feature!
  11. 11. GETTING STARTED
  12. 12. Server deployment exampleHost1• Coordinator• MongoDBHost2• CorrelatorHost3• Storage
  13. 13. HOST1• Creating a development environment[LINK]• Install MongoDB[LINK]
  14. 14. uWSGI.ini
  15. 15. meniscus/etc/meniscus/meniscus.conf
  16. 16. Run uwsgi daemon!!# uwsgi uwsgi.ini
  17. 17. HOST2• Creating a development environment[LINK]• Run uwsgi daemon!!Default Persona : pairing
  18. 18. CURL Request• Attempts pairing with coordinator• Becomes correlation worker after automatically restart
  19. 19. HOST3• Creating a development environment[LINK]• Run uwsgi daemon!!Default Persona : pairing
  20. 20. CURL Request• Attempts pairing with coordinator• Becomes store worker after automatically restart
  21. 21. COMPLETED FOR SETTING UP
  22. 22. Logging a data• Already MUST register a tenant• Request to correlator server(host2)
  23. 23. FlowCorrelator(host2)Storage(host3)MongoDB(host1)
  24. 24. Conclusion• Well-made backend APIs for logging(Good architecture)• Distributed workers processing• Diverse Persona(worker)• In early development• Nobody in its community(Stack Overflow, Google Groups)
  25. 25. Not yet• Data Store – Only MongoDBHadoop / HDFS / HbaseElastic Search• Perfect documentationneed more examplesneed exact examples
  26. 26. The thing we can now do with meniscus isJUST to create log data into mongodb viarestful api and to fetch log data frommongodb.
  27. 27. THANKS

×