Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Drawing parallels between tirupati and high scalability

A few thoughts on real world throttling

  • Be the first to comment

  • Be the first to like this

Drawing parallels between tirupati and high scalability

  1. 1. While writing my previous blog extremely-large-scale-web-applications-a-must-read-for-every-architect/,I hadplannedatrip to Tirupati. It was long due and my LONG trip away from home to Liverpool was scheduled to be a week after. My lastcoupleof visitswaswith friends andthiswasthe first visitwithChaitra (Wife) andRynia(my princess daughter), and secondvisit with my mom and dad. The first with them was a long time ago when I was a little boy. So I was super excited. What I did NOT realize was that, it was a Friday, with Lunar Eclipse the next day and the summer holidays had just started. On a Lunar Eclipse the temple was scheduled to be closed for over 11 hours. Being the most visited place of worship in the world, the usually 100,000 visitors a day seemingly was double that. It is usually good to book trips with agencies who organize the complete package. They obviously know what’sbest? NOTREALLY.... Tirumala now has booking online, which obviously I dintknow, theagency dint recommend thateither. Whenwe gotthere, they saidthe bestandthefastestway isvia theSarvaDarshan, the free darshan, without any pre-booking, that’s the only option. To top it all, the Chief Minister was visitingthe temple andthatonly meanslonger waiting time. . . . Wereached Tirupatiat3:30 am . . .and the wait began . . Waiting in the line, my bulwark against any kinds of phobia, had started to feel being crushed by demophobia, 100’s of 1000’s of visitors rushing their way through like particles accelerated by a force within, beingin alarge hadroncollider, I couldn’tbutstopthinkingabout thesheernumber, andthetemple authority(TTD: Tirumala Tirupati Devasthanams) quandary. Imagine processingthat many numberof requests - Cannotload balance, No denial of requests, Noerror codes, requests cannot be processed concurrently, No way to distribute requests and No special way to handle bursts in activity such as lunar eclipse Few measures had been taken by TTD though: Throttling, Throttling and more Throttling
  2. 2. Queuing requests – Needless to say Sarva Darshan queue was the longest There are different types of queues. Free queue (Sarva Darshan), Paid queues (multiple of them – All booked via online now) and queue where visitor climb up the hills via the steps And as we can expect, they are prioritized queues. Sarva Darshan having the lowest priority. While bits and pieces of the queue processing design patterns such as Throttling pattern, Priority Queue Pattern, Queue based load levelling pattern all seemed to have been applied here, It also reminded me of the Leaky bucketalgorithm, where visitorscome tothe queuelike a water being addedto the bucket, with a maximum request quota of few thousands beyond which is throttled with a restore rate of a hours. Yes Hours ! Rynia kept the trip alive and fun with her playing, skillfully using the sketch pen around my neck and Govinda, Govinda in between repeating after the piligrims It is time to introduce the VaikuntamQueueComplex (VQC). A 57 room facility with 3 levels and19 rooms on each level. After movingthroughthe standingqueue for hours, thevisitors are throttledin this facility. We reached this facility by 12:30 pm. We were made to wait in it till 10:00 pm, before the queue started to move again
  3. 3. When we came to thatroom, there was a boardwhich said thatthe darshanam (viewing God) is next day morning4:00 am. In front of us, was a life size television with live abhishekamindisplay. Justlike customersleaving website viewing somecached content, the wait time was so muchthat a few folks decided to leave. There was photocard systemin the buildingwhere in youcould get the cards, leave the buildingand get back later, however apartfrom the “4:00 am” signthere wasn’tanybody whowould/couldexplain the reasonfor the delay. Rememberno error codes, notime delay messages either. And that“4:00 am” messagewas acting like a proxy server/request filter redirecting messagesto the CDN with the cached requests, pushingvisitorsaway to come back to visit anotherday. All credit to TTD thoughtoensure thatno visitorstayshungry. Be it the standingqueue or in the VQC, frequently the team wouldcome by, to server food, snacksand all for free of cost.
  4. 4. The queue startedto move againat 10:30 pm. There was a small detour of further throttlingto collect the ladduprasadamtickets anda confluence of visitors from all the queues. Finally an hourafter we were endowed to be in front of the omnipresent. Dint fail to notice one more thing, the request accelerators (TTD staff), who will not let u stand beyond a second in front of the GOD I have to say, the power of faith is far greater than mostperceive. Even after hoursand hoursof standing, walking and waiting, My momwho has a nagging pain in her knees, dint feel the pain, and rather felt that the pain she had, has reduced, my daughter got her wishful toys, my wife and dad happy and myself relieved that the trip was safe, satisfied and venerational at different levels. .