NOMS 2004 JJ Paper
•Download as PPT, PDF•
1 like•334 views
Traffic Engineering in Cable-Data System : Maximizing CMTS port utilization
![Traffic Engineering in Cable-Data System : Maximizing CMTS port utilization Jun-seong Cho, Jongheon Park, Munhee Hong NMS Development Part, Research and Development Center Hanaro Telecom, Inc. Seoul Korea Email: [email_address] Tel: +82-26266-4244](https://image.slidesharecdn.com/noms2004-jj3-100314205709-phpapp01/85/NOMS-2004-JJ-Paper-1-320.jpg)
![Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],HFC > 2000 HOMES Headend N COAX Fiber Cable Serving Area](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
Similar to NOMS 2004 JJ Paper
Similar to NOMS 2004 JJ Paper (20)
Recently uploaded
Recently uploaded (20)
NOMS 2004 JJ Paper
- 1. Traffic Engineering in Cable-Data System : Maximizing CMTS port utilization Jun-seong Cho, Jongheon Park, Munhee Hong NMS Development Part, Research and Development Center Hanaro Telecom, Inc. Seoul Korea Email: [email_address] Tel: +82-26266-4244
- 3. Intro on HFC Network’s Cable Data System Core part of Cable Data System configuration are CMTS, Port, CM, Cell ※ CMTS: Cable Modem Termination System, CM: Cable Modem ※ a cell can be provisioned to be handled by several port Internet CM CM CM CELL CMTS ONU Node Element(tap,amplifier etc.) Optical Fiber Coaxial Cable
- 8. LCM Basic Idea – Port Load Balancing
- 13. Result – After applying LCM 6/0, After applying LCM 4/0, After applying LCM * Traffic computed by five minute interval sampling
Editor's Notes
- [Abstract] CMTS has played the key role in HFC’s cable-data system, and its port resources are the key factors that manage a operational group of subscribers. Usually a port is provisioned to manage a cell which covers the geographically grouped subscribers in HFC network. Since each upstream or downstream port has got a certain capacity, too many subscribers assigned on a CMTS port make all the subscribers suffer from bandwidth contention. That’s why service providers put a limit on per-cell subscribers(ie. 200 subscribers per cell). However, putting equal limit on the number of subscribers among ports has been proved not useful any longer. In my previous research[1], I have discussed the issue of upstream bandwidth contention. This time, the paper will discuss how to maximize the port utilization from the service provider’s viewpoint, especially by traffic load balancing. The method suggested is highly dynamical, economical, and has been implemented, field-tested. The paper will show not only the traffic engineering mechanism, but also the result of the test applying the mechanism.
- The popularity of internet has grown up to the extent of reaching every residence. People now not only requires connectivity but also higher bandwidth. One of the major broadband services is the Internet Access Service via HFC(Hybrid Fiber coaxial) network. According to a statistics issued by Korea Telecommunications Operators Association in ’02. May, the portion of the cable network Internet access users occupies 34% among total 9,076,299 Korean Internet access users, and it continues to increase. To the contrary of expectation that the increase of subscribers shall return better ROI, service providers’ ROI still has been getting nowhere. The speed of subscriber increase simply doesn’t match up with the one of traffic increase due to the advent of multimedia age. Exponentially increasing traffic asks for more network resources which are highly costly. Therefore, for the sake of best resource utilization, and cost-cutting operation, the paper introduces a very useful and effective traffic engineering scheme. It has been implemented and run in a real cable-data system environment, and proven very effective, and very much cost-saving. To explain ever-increasing resource shortage, a few basic concepts have to be explained first. As a matter of fact, there are a lot to know to understand broadband service using HFC network, such as physical HFC network components, HFC network architecture, Internet protocols, CMTS, subscriber devices, etc. In this paper, the core concepts such as CMTS, port, cell, CM, and extreme users are elaborated.
- CMTS, CM and port CMTS as the device that terminates signals from its subscriber’s modems(CM) bridges communication between user’s computer and the world of Internet. CMTS converts data from Internet into digital signals that are modulated for transmission over HFC plant, and then demodulated by the cable modem in the home or business. Cardinality has not been shown in Figure, but one CMTS port provides a dedicated 27 Mbps downstream data channel that is shared by the 500 to 1,000 homes served by a fiber node, or group of nodes. A single CMTS upstream port has the bandwidth per channel typically ranges from 2 Mbps to 10 Mbps. Cell Usually a port manages a cell, and a cell can be provisioned to be handled by several port. A cell covers the geographically grouped subscribers in HFC network. Cell, as an operational unit of geographical subscribers grouping, means a coverage that is served by a branch from a ONU in HFC network. Its network/service operator are responsibility to define and manage the cell. The figure describes a typical cell. CMTS, Port, Cell, CM Relationship between CMTS and HFC network are typically mapped to the one between CMTS port and cell. From operational perspective, mapping cells to ports is very excellent management scheme. By the mapping, operators can have full control of the device, and subscribers serviced.
- CMTS has many ports(upstream ports and downstream ports separately). The counter part of CMTS port on the user side is CM. Thus the cardinality between CMTS port and CM is one to many. As told, a single CMTS port is shared by the home-users who are provisioned to the port. Thus, there is possibility that a few monopolize the bandwidth. If it occurs, the port gets into the busy state. If it becomes busier to reach up to panic level, existing users get to experience extremely slow speed. Worse than that, the other users who belong to the same port, but currently not generating any traffic won’t be able to use Internet when they want. The bandwidth exhaustion has been illustrated as the above As shown item 1 in the figure, the 12 blocks represents the bandwidth that the CMTS port P can handle at once. At the time t , user A and B took up all the bandwidth. Especially user A takes up unfairly large bandwidth portion and made the rest suffer(We call him/her “excessive user, ie. EU ”). It is fair to say there’s none left available at the time t due to the excessive user. If there are any other users who want to use internet like C, or who were already using Internet, they will be unable to use it until the part of resource is released. If t lasts short, it might be bearable. When the t lasts long, user’s patience runs out. Besides, the bandwidth exhaustion becomes overload of the CMTS itself, and make the CMTS port busy giving CPU overload to the device. It is considered as a signal that it can soon endanger Cable-Data System operation.
- Since each upstream or downstream port has got a certain capacity, assigning too many subscribers on a CMTS port makes the subscribers suffer from bandwidth contention, and bandwidth contention reveals itself to subscribers as World-Wide-Wait. One of the typical ways to deal with bandwidth contention is to put a limit on per-cell subscribers (ie. 200 subscribers per downstream port). However, equal limit on the number of subscribers among ports is not a good operational method any longer. As mentioned in the introduction, new advent of multimedia age has produced various Internet usages. Amongst them, we have spotted a few users monopolizing the CMTS port bandwidth. To understand how much EU can be burdensome to resource utilization, we have analyzed a CMTS which runs on a real cable-data system environment. A CMTS has been chosen from the HFC network in Hanaro Telecom Inc., and the traffic has been monitored through ’03.3.1 14:50~’03. 3. 21 14:50’. As shown, EUs who are only 9% of total subscribers are taking up 81% of total traffic. People might say it is always natural for a few to take the majority in free competition environment, and that is quite true. However, if this kind of bandwidth occupation, especially in its peak time, harms resource utilization, leads to usage deadlock, it is high time to get the benefit of traffic engineering.
- What about the actual traffic trends of each port? The graphs have been produced from the same CMTS during analysis period. Both downstream ports of the 4/0 and 6/0 have the same maximum interface speed which is 25 Mbps. From operational experience, it is not possible to reach the maximum, and it has been observed that the port becomes very busy when it reaches approximately 80% of the interface speed. In case of the ports illustrated, 80 % means 20 Mbps. Let’s compare 4/0 and 6/0. The number of subscribers are nearly same, and the port actually has been provisioned to the same cell. The following are the things that have been discovered during the observation Traffic in 4/0 seems to be ok except the time between 23:00~24:00 Traffic in 6/0 reaches the busy port level almost half the day. The user complaints in 6/0 have been piled up due to its slow network they experienced. (Some users are reported that they could not even use Internet during the time) It would be nice if 4/0 can spare some of its free bandwidth to 6/0 when 6/0 becomes busy.
- The aforementioned can be solved by the following three methods. Purchases enough network resources Lowers per-cell subscribers Engineers traffic, and minimize the inconvenience until new resources are added. First two solutions are simple but unrealistic. The method is not only uneconomical but also will make service provider’s ROI suffer. Thus, traffic engineering could be the only option, if it solves afore mentioned problems. The paper introduces a module called “ Load Control Manager (LCM)” and the LCM will monitor the traffic, and is responsible for the traffic engineering The next four slides will explain how LCM works.
- As shown, the basic idea is simply balancing the heavy load with other network resources. Let’s say the downstream ports in 4/0 and 6/0 has been provisioned to the same cell, and traffic of 4/0 has reached busy port level for some time. At the same time, you spotted traffic of 6/0 is not much and has got plenty of room to handle more. Now you will be thinking, “Why don’t I move some traffic from 4/0 to 6/0, and give a break to the busy one, 4/0?” To make the idea become real, there are some issues to be resolved. How can I know when a port is busy, and which is busy? When is the right time to balance load? How can we move the traffic from one to another, in other words, which EU’s traffic to move? What if there is no other ports to give the busy one a break The following sections will answer the issues, and finally shows what result has been produced by the idea.
- The first thing to do is to find the busy port. Since there is no need to worry if the port has got enough resource to handle CM traffics. If the bandwidth is ok, let EUs be. EU ’s being threatening is only the time when port gets busy, so that it can make the others suffer. Well defining “busy port” condition is very important. It is the status that can endanger the service in no time. When it becomes busy, usually the traffic reaches near to its maximum bandwidth. The performance record of the port can be the best indicator that tells whether it is busy or not. Then, what is the definition of port’s being “busy”? We learned the term “busy port” from field operators. In manual way, operators used to scan port status in the peak time. They looked up a certain performance status such as the number of users online, port utilization, bandwidth utilization, and realized that the port gets busy when each performance record reaches certain values. They define the thresholds of them, and the thresholds are the key to determine whether port is busy or not. Usually NMS performance management functionalities keep track of such values. Thus, it can let LCM be informed of it. Utilizing the NMS functionality enabled perfect and real time busy port detection. Thanks to it, not a single busy port go unnoticed, and operators don’t need to worry about busy port detection. The NMS performance management functionalities discovers a busy port, it gives LCM the following information Busy Port Information : CMTS ID(CMTS IP), Port Index, Port Usage CMTS Connection Info such as SNMP read community or telnet login id/password
- Since we know the port that could be overloaded, it is time to find EU. LCM connect to the CMTS where the busy port belongs, scan all the user status on the busy port, and f inally draws EU’s CM Mac address, average traffic that has been detected. LCM uses the following info to detect EU. Busy Port Information : CMTS ID(CMTS IP), Port Index, Port Usage CMTS Connection Info such as SNMP read community or telnet login id/password EU detection rules and policies : EU upstream / downstream threshold, Detection period, Port busy status before and after the monitoring, etc. Amongst the information LCM uses, the most important factor is to find EU upstream / downstream thresholds, Detection period. If EU thresholds are too low, then so many EUs will be detected so that there’s no point of management. EU thresholds should be not too high, not too low. Also it goes without saying detection period is not exception. It should be long enough to assume that the user is continuously generating the extreme traffic. If it is too long, busy port can reach up to a panic mode. ※ How to query the user traffic information from CMTS? There are two implementation methods that could be applied. First one is SNMP. SNMP is good because it is a standardized way of managing network device. However, using SNMP could be quite burdensome because the functionality requires retrieving each and every user’s traffic counter values on a given port. It leads to numerous SNMP getNext packets. The other way is to use the proprietary command line interface. In case of CISCO CMTS, operators can retrieve all the individual users’ traffic counters at once using one simple CLI command. Thus, in case of a single vender device network, CLI is less burdensome to both CMTS and LCM. [5][6] However, it becomes a different story if there are many different types of CMTS devices. CLIs are likely to differ according to the vender, and may not have the proper functionalities.
- Up to now, busy port is detected. Thus we know the information of the port such as utilization, bandwidth utilization. From the EU detection, we also know information on EUs such as id, traffic info, service category etc. Now, it is time to give the busy port a break. But who is going to share the load? The following steps tells about it. 1) Find the port that can handle the same cell. It means you can not move the traffic from the busy one to the port if the port is provisioned to handle the same cell. 2) Amongst the ports that can handle the cell, see if there is any port that can handle the EUs traffic(Refer to the simplified algorithm in the slide above). One thing which is very important is that the another port has got enough free bandwidth to handle – enough to cover - the EU’s traffic. If the traffic is moved to the port which is near to busy port level, then the port becomes busy right away so there would be no point of doing this scheme. 3) If port X, and EUs to move were found, LCM moves the user from the previous port to the port X. The relocation method can differ according to the CMTS device. In case of Cisco UBR 7000 series, we used Cisco proprietary CLI command. 4) Log and save the record for the sake of tracking afterwards. Other traffic Load Controlling Mechanism It is still possible there is no other port to handle the traffic. To avoid the panic situation where a lot of ordinary users can not use Internet owing to a few EUs, service provider might opt to limit the EUs bandwidth rate only for the busy time. Or, it might decide to bring in many EUs spread into several ports, and drive them to a single port, so ordinary users get no harm from them. We have experimented the method in a small area for a short period of time, and found out that the ordinary users who are taking majority are very happy about the improved quality. However, some of the EUs who are relatively a few and gathered in the same port, began to complain the slowed speed. The Other traffic Load Controlling Mechanism can vary according to the company’s policy. It also can have a great impact on SLAs. Thus the research can be furthered.
- Researching this mechanism, we have realized that setting the right thresholds makes all different. As already told, t hresholds always becomes the bottom line of the mechanism. Threshold can differ depending on the service provider’s environment. Thus, it is quite ok that different service providers have different threshold. What’s important on this is to make sure the values tested, analyzed, and carefully chosen. Think about busy port related threshold. Since all the sequence of the mechanism starts from busy port detection, wrong threshold can give another burden to both CMTS and LCM management system. As far as threshold values are concerned, various combinations are possible, but usually the related threshold will be port status, port utilization, port alarm record, channel utilization, etc. EU thresholds are as much important as busy port threshold. Once ill set, it can make everybody be EU. If EUs are majority, then they are not EU any more. EU related thresholds that have been found here are EU upstream/downstream thresholds and sampling period.(To calculate traffic, LCM has to sample twice the octet counter value of a user. The time between two sampling moments is called sampling period) Provided that all the threshold set correct, still there are things to be solved further to apply in the running environment. For example, the EU will lose the connection temporarily while port relocation. EU traffic controlling mechanisms such as EU relocation, rate limiting may violate the SLA. Also, EUs after relocated may experience slower speed than before especially in the peak time. Thus, applying the method can be illegal to the user who are guaranteed to have a best-effort all time connection. Regarding those, special care has to be taken before applying the mechanism.
- After applying LCM, we have observed the traffic once again on the same ports. The graphs above have been the outcome of it. Firstly 4/0 seems to remain the same as before, or seems that the traffic has increased a little more. However, the graph of 6/0 has changed dramatically. Before LCM, the port has been busy nearly half of the day. However, it seems quite ok all day now. In fact, the subscriber complaints also decreased. How can it be possible? As expected, when port becomes busy, LCM managed to find the EUs. and relocated it to another port which is either 4/0 or 6/0. The EU who is just relocated will set up a new connection at the new port. But this time, s/he may not be able to use the same amount of bandwidth, because already others have taken up the bandwidth quite much, and his/her application may have to begin its slow start again. Besides if s/he tries to monopolize the bandwidth again, LCM may relocate the EUs again. The result we saw definitely seems to be more stable and safe than before. The mechanism achieved highly efficient traffic load balance management. The mechanism suggested has been implemented, tested in a field. It has been proven flexible, efficient in a real cable-data system environment.
- EU’s monopolization of CMTS port resource is quite dangerous. If the case turns out worst, it can endanger the Internet service itself of the port. To solve the threat, the paper has suggested a traffic engineering scheme, called LCM. Thanks to LCM, EUs have been identified, managed well, and port utilization has been improved. However, to apply the scheme for service provider’s environment in general, there are things to be solved as shown above. Further study on the issues should be followed up for better use. References [1] Jun-Seong Cho, “Efficient Upstream Management in Cable-Data System: Weaving OSS, NMS, and Authentication System”, session I-3 APNOM 2003 Proceedings, Asia-Pacific Network Operations and Management, 2003. [2] Jun-Seong Cho, “Effective Upstream Management in cable-data system”, session III-3 KNOM 2003 Proceedings, Korean Network Operations and Management, 2003. [3] KTOA(Korea Telecommunications Operators Association), Numbers on Korean broadband Internet subscribers - (2002.5), http://www.ktoa.or.kr/ktoa_data/speed_list.shtml, 2002. 5. [4] Jun-Seong Cho, Jong-Hun Park, Seong-Bong Hong, “CATV Regional Network Management System”, pp. 123-124, Hanaro Telecom Inc., R&D center, 2002 [5] Cisco Systems, Inc., P.E., “Migrating Simple Data Over Cable Services DOCSIS 1.1,” Cisco System, Inc.., pp. 57-58, 2002. [6] Hanaro Telecom., Kim, Lee, “Field Operation Manual for CATV/HFC broadband services,” Hanaro Telecom Inc. 2001.10.16 ~10.20 Training Session, Chapter. Cisco Router Basic pp. 8-11, 2002 10. [7] Seong-bong Hong, “ Auto recovery algorithm of HFC network fault based on statistic model “ , session I-1 KNOM 2003 Proceedings, Korean Network Operations and Management, 2003.