Logging rails application behavior to MongoDB

2,033 views

Published on

Logging rails application behavior to MongoDB

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,033
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Logging rails application behavior to MongoDB

  1. 1. Logging rails application behavior to MongoDB Vasiliev Alexey Yuriyovich leopard.in.ua 2011 Vasiliev A.Y. Logging rails application behavior to MongoDB
  2. 2. MongoDB Introduction MongoDB is an open source document-oriented database. Features Consistent UTF-8 encoding; Cross-platform support; Efficient storage for large binary data, for example photos and video; Replication and failover; MapReduce. Vasiliev A.Y. Logging rails application behavior to MongoDB
  3. 3. Why application should log to DB? Centralized application logging; Easy remote access, compared to files; Can be fast and scalable if you go with database partitioning and/or multi-log databases. Vasiliev A.Y. Logging rails application behavior to MongoDB
  4. 4. What are the advantages appear logging in MongoDB? Flexible schema; Asynchronous inserts; Capped collection; Fast enough compared to files; Indexed and searched; Analyze data in-place. Vasiliev A.Y. Logging rails application behavior to MongoDB
  5. 5. Gem for RailsMongoDB as Rails logger Central Logger (https://github.com/ customink/central_logger). You should do only several steps: gem ’central_logger’; include CentralLogger::Filter; Config the connection to mongodb; That’s all! Vasiliev A.Y. Logging rails application behavior to MongoDB
  6. 6. Rails log structureMongoDB as Rails logger { ’ action ’ : action_name , ’ application_name ’ : application_name ( r a i l s root ) , ’ controller ’ : controller_name , ’ ip ’ : ip_address , ’ messages ’ : { ’ info ’ : [ ], ’ debug ’ : [ ] , ’ error ’ : [ ] , ’ warn ’ : [ ], ’ fatal ’ : [ ] }, ’ params ’ : { }, ’ path ’ : path , ’ request_time ’ : date_of_request , ’ runtime ’ : execution_time_in_milliseconds , ’ url ’ : full_url } Vasiliev A.Y. Logging rails application behavior to MongoDB
  7. 7. Add data to logMongoDB as Rails logger Add additional data to log # make s u r e we ’ r e u s i n g t h e C e n t r a l L o g g e r i n t h i s e n v i r o n m e n t i f R a i l s . l o g g e r . r e s p o n d _ t o ? ( : add_metadata ) R a i l s . l o g g e r . add_metadata ( : u s e r _ g u i d => @ u s e r . i d ) end Vasiliev A.Y. Logging rails application behavior to MongoDB
  8. 8. Querying via the Rails consoleMongoDB as Rails logger Handle on the MongoDB >> db = R a i l s . l o g g e r . mongo_connection => #&l t ; Mongo : : DB: 0 x 1 0 2 f 1 9 a c 0 @ s l a v e _ o k= n i l ... >> c o l l e c t i o n = db [ R a i l s . l o g g e r . m o n g o _ c o l l e c t i o n _ n a m e ] => #&l t ; Mongo : : C o l l e c t i o n : 0 x 1 0 3 1 b 3 e e 8 . . . Find all requests for a specific user (with guid) >> c u r s o r = c o l l e c t i o n . f i n d ( : u s e r _ g u i d => 1 2 3 5 5 ) => #&l t ; Mongo : : C u r s o r : 0 x 1 0 3 1 a 3 e 3 0 . . . &g t ; >> c u r s o r . count => 255 Vasiliev A.Y. Logging rails application behavior to MongoDB
  9. 9. Querying via the Rails consoleMongoDB as Rails logger Find all requests that took more that two second to complete >> c o l l e c t i o n . f i n d ( { : r u n t i m e => { ’ $gt ’ => 2 0 0 0 } } ) . c o u n t => 3 Find all requests with an exception >> c o l l e c t i o n . f i n d ( { " m e s s a g e s . e r r o r " => / R o u t i n g E r r o r / } ) Vasiliev A.Y. Logging rails application behavior to MongoDB
  10. 10. Querying via the Rails consoleMongoDB as Rails logger Find by params values >> c o l l e c t i o n . f i n d ( " params . a c t i v i t y " => { ’ $ e x i s t s ’ => t r u e } , " params . a c t i v i t y . s t o r i e s " => { ’ $ e x i s t s ’ => t r u e } , " r e q u e s t _ t i m e " => { ’ $ g t e ’ => s t a r t _ d a t e . u t c , ’ $ l t e ’ => end_date . u t c } ) . s o r t ( " params . a c t i v i t y . o c c u r r e d _ a t " , 1 ) . c o u n t => 542 Vasiliev A.Y. Logging rails application behavior to MongoDB
  11. 11. Querying via the Rails consoleMongoDB as Rails logger Find current status of story using logs >> c o l l e c t i o n . f i n d ( " params . a c t i v i t y . s t o r i e s . i d " => s t o r y _ i d . to_i , " params . a c t i v i t y . s t o r i e s . c u r r e n t _ s t a t e " => { ’ $ e x i s t s ’ => t r u e } ) . s o r t ( " params . a c t i v i t y . o c c u r r e d _ a t " , −1). f i r s t => {" _id"=>BSON : : O b j e c t I d ( ’ 4 e 7 c 6 2 e d 7 6 1 2 6 8 0 4 d 4 0 0 0 0 0 f ’ ) , " a c t i o n "=>"hook " , " c o n t r o l l e r "=>" s e r v i c e s " , " p a t h "=>"/ a p i / v1 / hook " , " params"=>{" a c t i v i t y "=>{ . . . , " s t o r i e s " = > [ { . . . , " c u r r e n t _ s t a t e "=>" f i n i s h e d " } ] } } Vasiliev A.Y. Logging rails application behavior to MongoDB
  12. 12. Querying via the Rails consoleMongoDB as Rails logger Tailable Cursors v a r c o l l = db . some . c a p p e d . c o l l e c t i o n ; var l a s t V a l = c o l l . f i n d ( ) . s o r t ({ ’ $ n a t u r a l ’ : 1 }) . l i m i t ( 1 ) . next ( ) [ ’ increasing ’ ]; while (1){ c u r s o r = c o l l . f i n d ({ ’ i n c r e a s i n g ’ : { ’ $gte ’ : l a s t V a l } }); // t a i l a b l e c u r s o r . addOption ( 2 ) ; // a w a i t d a t a c u r s o r . a d d O p t i o n ( 32 ) ; // W a i t s s e v e r a l s e c f o r more d a t a w h i l e ( c u r s o r . hasNext ( ) ){ v a r doc = c u r s o r . n e x t ( ) ; l a s t V a l = doc [ ’ i n c r e a s i n g ’ ] ; p r i n t j s o n ( doc ) ; } } Vasiliev A.Y. Logging rails application behavior to MongoDB
  13. 13. Using MapReduceMongoDB as Rails logger Find average time of page loading >> c o l l e c t i o n . g r o u p ( " f u n c t i o n ( x ) { r e t u r n { month : x . r e q u e s t _ t i m e . getMonth ( ) , year : x . request_time . g e t F u l l Y e a r ( ) } ; }" , { : r e q u e s t _ t i m e => { ’ $ g t e ’ => Time . u t c ( 2 0 1 1 , 0 1 , 0 1 , 0 0 , 0 0 , 0 0 ) , " $ l t " => Time . u t c ( 2 0 1 1 , 8 , 0 1 , 0 0 , 0 0 , 0 0 ) } } , { : c o u n t => 0 , : t o t a l _ t i m e => 0 } , " f u n c t i o n ( doc , o u t ) { o u t . c o u n t ++; o u t . t o t a l _ t i m e+=doc . r u n t i m e } " , " f u n c t i o n ( out ){ o u t . avg_time = o u t . t o t a l _ t i m e / o u t . c o u n t } " ) => [ {" month "= >4.0 , " y e a r "= >2011.0 , " c o u n t "= >5385.0 , " t o t a l _ t i m e "= >1027015.0 , " avg_time "= >190.717734447539} , ... ] Vasiliev A.Y. Logging rails application behavior to MongoDB
  14. 14. RestrictionsMongoDB as Rails logger The database doesn’t allow update the existing objects in the collection (but there is possibility). The database doesn’t allow deleting objects from a capped collection. Capped collection are not shardable. More indexes - slower inserts. Vasiliev A.Y. Logging rails application behavior to MongoDB
  15. 15. Q&A Thank you for attention! Vasiliev A.Y. Logging rails application behavior to MongoDB

×