Ultra-Low Power Electronics and Design
Upcoming SlideShare
Loading in...5

Ultra-Low Power Electronics and Design



This book appears at a time when the first examples of complex circuits in 65nm CMOS technology are beginning to appear, and these products already must take advantage of many of the techniques to be ...

This book appears at a time when the first examples of complex circuits in 65nm CMOS technology are beginning to appear, and these products already must take advantage of many of the techniques to be discussed and developed in this book. So why then should our increasing success at miniaturization, as evidenced by the success of
Moore’s Law, be creating so many new difficulties in power management in
circuit designs?



Total Views
Views on SlideShare
Embed Views



3 Embeds 11

http://www.linkedin.com 6
http://www.techgig.com 4
https://www.linkedin.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Ultra-Low Power Electronics and Design Ultra-Low Power Electronics and Design Document Transcript

  • This page intentionally left blank
  • Ultra Low-PowerElectronics and Design Edited by Enrico Macii Politecnico di Torino, Italy KLUWER ACADEMIC PUBLISHERS NEW YORK, BOSTON, DORDRECHT, LONDON, MOSCOW
  • eBook ISBN: 1-4020-8076-XPrint ISBN: 1-4020-8075-1©2004 Springer Science + Business Media, Inc.Print ©2004 Kluwer Academic PublishersDordrechtAll rights reservedNo part of this eBook may be reproduced or transmitted in any form or by any means, electronic,mechanical, recording, or otherwise, without written consent from the PublisherCreated in the United States of AmericaVisit Springers eBookstore at: http://www.ebooks.kluweronline.comand the Springer Global Website Online at: http://www.springeronline.com
  • viiContributorsA. Acquaviva Università di UrbinoL. Benini Università di BolognaD. Bertozzi Università di BolognaD. Blaauw University of Michigan, Ann ArborA. Bogliolo Università di UrbinoA. Bona STMicroelectronicsC. Brandolese Politecnico di MilanoW.C. Cheng University of Southern CaliforniaG. De Micheli Stanford UniversityN. Dutt University of California, IrvineW. Fornaciari Politecnico di MilanoF. Gaffiot Ecole Centrale de LyonJ. Gautier CEA-DRT–LETI/D2NT–CEA/GREA. Gordon-Ross University of California, RiversideR. Gupta University of California, San DiegoC. Heer Infineon Technologies AGM. J. Irwin Pennsylvania State UniversityI. Kadayif Canakkale Onsekiz Mart UniversityM. Kandemir Pennsylvania State UniversityB. Kienhuis LeidenI. Kolcu UMISTE. Lattanzi Università di UrbinoD. Lee University of Michigan, Ann ArborA. Macii Politecnico di TorinoS. Mohapatra University of California, IrvineI. O’Connor Ecole Centrale de LyonK. Patel Politecnico di TorinoM. Pedram University of Southern CaliforniaC. Pereira University of California, San DiegoC. Piguet CSEMM. Poncino Università di VeronaF. Salice Politecnico di MilanoP. Schaumont University of California, Los AngelesU. Schlichtmann Technische Universität MünchenD. Sylvester University of Michigan, Ann Arbor
  • viiiF. Vahid University of California, Riverside and University of California, IrvineN. Venkatasubramanian University of California, IrvineI. Verbauwhede University of California, Los Angeles and K.U.LeuvenN. Vijaykrishnan Pennsylvania State UniversityV. Zaccaria STMicroelectronicsR. Zafalon STMicroelectronicsB. Zhai University of Michigan, Ann ArborC. Zhang University of California, Riverside
  • ixPrefaceToday we are beginning to have to face up to the consequences of thestunning success of Moore’s Law, that astute observation by Intel’s GordonMoore which predicts that integrated circuit transistor densities will doubleevery 12 to 18 months. This observation has now held true for the last 25years or more, and there are many indications that it will continue to holdtrue for many years to come. This book appears at a time when the firstexamples of complex circuits in 65nm CMOS technology are beginning toappear, and these products already must take advantage of many of thetechniques to be discussed and developed in this book. So why then shouldour increasing success at miniaturization, as evidenced by the success ofMoore’s Law, be creating so many new difficulties in power management incircuit designs? The principal source and the physical origin of the problem lies in thedifferential scaling rates of the many factors that contribute to powerdissipation in an IC – transistor speed/density product goes up faster thanthe energy per transition comes down, so the power dissipation per unit areaincreases in a general sense as the technology evolves. Secondly, the “natural” transistor switching speed increase from onegeneration to the next is becoming downgraded due to the greater parasiticlosses in the wiring of the devices. The technologists are offsetting thisproblem to some extent by introducing lower permittivity dielectrics (“low-k”) and lower resistivity conductors (copper) – but nonetheless to get theneeded circuit performance, higher speed devices using techniques such assilicon-on-insulator (SOI) substrates, enhanced carrier mobility (“strainedsilicon”) and higher field (“overdrive”) operation are driving powerdensities ever upwards. In many cases, these new device architectures areincreasingly leaky, so static power dissipation becomes a major headache inpower management, especially for portable applications.
  • x A third factor is system or application driven – having all this integrationcapability available encourages us to combine many different functionalblocks into one system IC. This means that in many cases, a large part of thechip’s required functionality will come from software executing on andbetween multiple on-chip execution units; how the optimum partitioningbetween hardware architecture and software implementation is obtained is avast subject, but clearly some implementations will be more energy efficientthan others. Given that, in many of today’s designs, more than 50% of thetotal development effort is on the software that runs on the chip, getting thispartitioning right in terms of power dissipation can be critical to the successof (or instrumental in the failure of!) the product. A final motivation comes from the practical and environmentalconsequences of how we design our chips – state-of-the-art highperformance circuits are dissipating up to 100W per square centimeter – weonly need 500 square meters of such silicon to soak up the output of a smallnuclear power station. A related argument, based on battery lifetime, showsthat the “converged” mobile phone application combining telephony, datatransmission, multimedia and PDA functions that will appear shortly isdemanding power at the limit of lithium-ion or even methanol-water fuel cellbattery technology. We have to solve the power issue by a combination ofdesign and process technology innovations; examples of current approachesto power management include multiple transistor thresholds, triple gateoxide, dynamic supply voltage adjustment and memory architectures. Multiple transistor thresholds is a technique, practiced for several yearsnow, that allows the designer to use high performance (low Vt) deviceswhere he needs the speed, and low leakage (high Vt) devices elsewhere. Thisbenefits both static power consumption (through less sub-threshold leakage)and dynamic power consumption (through lower overall switching currents).High threshold devices can also be used to gate the supplies to different partsof the circuit, allowing blocks to be put to sleep until needed. Similar to the previous technique, triple gate oxide (TGO) allows circuitpartitioning between those parts that need performance and other areas of thecircuit that don’t. It has the additional benefit of acting on both sub-thresholdleakage and gate leakage. The third oxide is used for I/O and possiblymixed-signal. It is expected over the next few years that the processtechnologists will eventually replace the traditional silicon dioxide gatedielectric of the CMOS devices by new materials such as rare earth oxideswith much higher dielectric constants that will allow the gate leakageproblem to be completely suppressed.
  • xi Dynamic supply voltage adjustment allows the supply voltage to differentblocks of the circuit to be adjusted dynamically in response to the immediateperformance needs for the block – this very sophisticated technique will takesome time to mature. Finally, many, if not most, advanced devices use very large amounts ofmemory for which the contents may have to be maintained during standby;this consumes a substantial amount of power, either through refreshingdynamic RAM or through the array leakage for static RAM. Traditional non-volatile memories have writing times that are orders of magnitude too slowto allow them to substitute these on-chip memories. New developments,such as MRAM, offer the possibility of SRAM-like performance coupledwith unlimited endurance and data retention, making them potentialcandidates to replace the traditional on-chip memories and remove thiscomponent of standby power consumption. Most of the approaches to power management described briefly abovewill be employed in 65nm circuits, but there are a lot more good ideaswaiting to be applied to the problem, many of which you will find clearlyand concisely explained in this book. Mike Thompson, Philippe Magarshack STMicroelectronics, Central R&D Crolles, France
  • This page intentionally left blank
  • xiiiIntroductionULTRA LOW-POWER ELECTRONICS ANDDESIGNEnrico MaciiPolitecnico di TorinoPower consumption is a key limitation in many electronic systems today,ranging from mobile telecom to portable and desktop computing systems,especially when moving to nanometer technologies. Power is also ashowstopper for many emerging applications like ambient intelligence andsensor networks. Consequently, new design techniques and methodologiesare needed to control and limit power consumption.The 2004 edition of the DATE (Design Automation and Test in Europe)conference has devoted an entire Special Focus Day to the power problemand its implications on the design of future electronic systems. In particular,keynote presentations and invited talks by outstanding researchers in the fieldof low-power design, as well as several technical papers from the regularconference sessions have addressed the difficulties ahead and advancedstrategies and principles for achieving ultra low-power design solutions.Purpose of this book is to integrate into a single volume a selection of thesecontributions, duly extended and transformed by the authors into chaptersproposing a mix of tutorial material and advanced research results.The manuscript consists of a total of 14 chapters, addressing different aspectsof ultra low-power electronics and design. Chapter 1 opens the volume byproviding an insight to innovative transistor devices that are capable ofoperating with a very low threshold voltage, thus contributing to a significantreduction of the dynamic component of power consumption. Solutions forlimiting leakage power during stand-by mode are also discussed. The chaptercloses with a quick overview of low-power design techniques applicable atthe logic level, including multi-Vdd, multi-Vth and hybrid approaches.Chapter 2 focuses on the problem of reducing power in the interconnectnetwork by investigating alternatives to traditional metal wires. In fact,according to the 2003 ITRS roadmap, metallic interconnections may not beable to provide enough transmission speed and to keep power under controlfor the upcoming technology nodes (65nm and below). A possible solution,explored in the chapter, consists of the adoption of optical interconnectnetworks. Two applications are presented: Clock distribution and datacommunication using wavelength division multiplexing.
  • xivIn Chapter 3, the power consumption problem is faced from the technologypoint of view by looking at innovative nano-devices, such as single-electronor few-electron transistors. The low-power characteristics and potential ofthese devices are reviewed in details. Other devices, including carbon nano-tube transistors, resonant tunnelling diodes and quantum cellular automataare also treated.Chapter 4 is entirely dedicated to advanced design methodologies forreducing sub-threshold and gate leakage currents in deep-submicron CMOScircuits by properly choosing the states to which gates have to be drivenwhen in stand-by mode, as well as the values of the threshold voltage and ofthe gate oxide thickness. The authors formulate the optimization problem forsimultaneous state/Vth and state/Vth/Tox assignments under delay constraintsand propose both an exact method for its optimal solution and two practicalheuristics with reasonable run-time. Experimental results obtained on anumber of benchmark circuits demonstrate the viability of the proposedmethodology.Chapter 5 is concerned with the issue of minimizing power consumption ofthe memory subsystem in complex, multi-processor systems-on-chip(MPSoCs), such as those employed in multi-media applications. The focus ison design solutions and methods for synthesizing memory architecturescontaining both single-ported and multi-ported memory banks. Powerefficiency is achieved by casting the memory partitioning design paradigm tothe case of heterogeneous memory structures, in which data need to beaccessed in a shared manner by different processing units.Chapter 6 addresses the relevant problem of minimizing the power consumedby the cache hierarchy of a microprocessor. Several design techniques arediscussed, including application-driven automatic and dynamic cacheparameter tuning, adoption of configurable victim buffers and frequent-valuedata encoding and compression.Power optimization for parallel, variable-voltage/frequency processors is thesubject of Chapter 7. Given a processor with such an architecture, thischapter investigates the energy/performance tradeoffs that can be spanned inparallelizing array-intensive applications, taking into account the possibilitythat individual processing units can operate at different voltage/frequencylevels. In assigning voltage levels to processing units, compiler analysis isused to reveal hetherogeneity between the loads of the different units inparallel execution.
  • xvChapter 8 provides guidelines for the design and implementation of DSP andmulti-media applications onto programmable embedded platforms. TheRINGS architecture is first introduced, followed by a detailed discussion onpower-efficient design of some of the platform components, namely, theDSPs. Next, design exploration, co-design and co-simulation challenges areaddressed, with the goal of offering to the designers the capability ofincluding into the final architecture the right level of programmability (orreconfigurability) to guarantee the required balance between systemperformance and power consumption.Chapter 9 targets software power minimization through source codeoptimization. Different classes of code transformations are first reviewed;next, the chapter outlines a flow for the estimation of the effects that theapplication of such transformations may have on the power consumed by asoftware application. At the core of the estimation methodology there is thedevelopment of power models that allow the decoupling of processor-independent analysis from all the aspects that are tightly related to processorarchitecture and implementation. The proposed approach to software powerminimization is validated through several experiments conducted on anumber of embedded processors for different types of benchmarkapplications.Reduction of the power consumed by TFT liquid crystal displays, such asthose commonly used in consumer electronic products is the subject ofChapter 10. More specifically, techniques for reducing power consumptionof transmissive TFT-LCDs using a cold cathode fluorescent lamp backlightare proposed. The rationale behind such techniques is that the transmittancefunction of the TFT-LCD panel can be adjusted (i.e., scaled) while meetingan upper bound on a contrast distortion metric. Experimental results showthat significant power savings can be achieved for still images with very littlepenalty in image contrast.Chapter 11 addresses the issue of efficiently accessing remote memoriesfrom wireless systems. This problem is particularly important for devicessuch as palmtops and PDAs, for which local memory space is at a premiumand networked memory access is required to support virtual memoryswapping. The chapter explores performance and energy of networkswapping in comparison with swapping on local microdrives and FLASHmemories. Results show that remote swapping over power-manageablewireless network interface cards can be more efficient than local swappingand that both energy and performance can be optimized by means of power-aware reshaping of data requests. In other words, dummy data accesses canbe preemptively inserted in the source code to reshape page requests in orderto significantly improve the effectiveness of dynamic power management.
  • xvi Chapter 12 focuses on communication architectures for multi-processor SoCs. The network-on-chip (NoC) paradigm is reviewed, touching upon several issues related to power optimization of such kinds of communication architectures. The analysis goes on a layer-by-layer basis, and particular emphasis is given to customized, domain-specific networks, which represent the most promising scenario for communication-energy minimization in multi-processor platforms. Chapter 13 provides a natural follow up to the theory of NoCs covered in the previous chapter by describing an industrial application of this type of communication architecture. In particular, the authors introduce an innovative methodology for automatically generating the power models of a versatile and parametric on-chip communication IP, namely the STBus by STMicroelectronics. The methodology is validated on a multi-processor hardware platform including four ARM cores accessing a number of peripheral targets, such as SRAM banks, interrupt slaves and ROM memories. The last contribution, offered in Chapter 14, proposes an integrated end-to- end power management approach for mobile video streaming applications that unifies low-level architectural optimizations (e.g., CPU, memory, registers), OS power-saving mechanisms (e.g., dynamic voltage scaling) and adaptive middleware techniques (e.g., admission control, trans-coding, network traffic regulation). Specifically, interaction parameters between the different levels are identified and optimized to achieve a reduction in the power consumption. Closing this introductory chapter, the editor would like to thank all the authors for their effort in producing their outstanding contributions in a very short time. A special thank goes to Mike Thompson and Philippe Magarshack of STMicroelectronics for their keynote presentation at DATE 2004 and for writing the foreword to this book. The editor would also like to acknowledge the support offered by Mark De Jongh and the Kluwer staff during the preparation of the final version of the manuscript. Last, but not least, the editor is grateful to Agnieszka Furman for taking care of most of the “dirty work” related to book editing, paging and preparation of the camera-ready material.
  • 1Chapter 1ULTRA-LOW-POWER DESIGN: DEVICE ANDLOGIC DESIGN APPROACHESChristoph Heer1 and Ulf Schlichtmann21 Infineon Technologies AG; 2Technische Universität MünchenAbstract Power consumption increasingly is becoming the bottleneck in the design of ICs in advanced process technologies. We give a brief introduction into the major causes of power consumption. Then we report on experiments in an advanced process technology with ultra-low threshold voltage (Vth) devices. It turns out that in contrast to older process technologies, this approach increasingly is becoming less suitable for industrial usage in advanced process technologies. Following, we describe methodologies to reduce power consumption by optimizations in logic design, specifically by utilizing multiple levels of supply voltage Vdd and threshold voltage Vth. We evaluate them from an industrial product development perspective. We also give a brief outlook to proposals on other levels in the design flow and to future work.Keywords: Low-power design, dynamic power reduction, leakage power reduction, ultra- low-Vth devices, multi-Vdd, multi-Vth, CVS1.1 INTRODUCTION The progress of silicon process technology marches on relentlessly. Aspredicted by Gordon Moore decades ago, silicon process technologycontinues to achieve improvements at an astonishing pace [1]. The numberof transistors that can be integrated on a single IC approximately doublesevery 2 years [2,3]. This engineering success has created innovative newindustries (e.g. personal computers and peripherals, consumer electronics)and revolutionized other industries (e.g. communications). Today, however, it is becoming increasingly difficult to achieveimprovements at the pace that the industry has become accustomed to. Moreand more technical challenges appear that require increasing resources to be
  • 2 solved [4]. One such problem is the increasing power consumption of integrated circuits. It becomes even more critical as an increasing number of today’s high-volume consumer products are battery-powered. In the following, we will consider the sources of power consumption and their development over time. We will show why reduction of power consumption increasingly is becoming critical to product success and will review traditional approaches in Sections 1.1 and 1.2. In Section 1.3 we will then analyze a potential solution based on introduction of an optimized transistor with a very low threshold voltage Vth. Thereafter, we will present and discuss logic-level design optimizations for power reduction in Section 1.4. Also, we will briefly point out potential optimizations on higher levels. Our observations are made from the perspective of industrial IC product development where technical optimizations must be carefully evaluated against the cost associated with achieving and implementing them. Mostly, the presented methodologies are already being utilized in leading-edge industrial ICs. 1.2 POWER CONSUMPTION BECOMES CRITICAL Depending on the type of end-product and its application, different aspects of power consumption are the primary concern: dynamic power or leakage power. Reduction of dynamic power consumption is a concern for almost all IC products today. For battery-powered products, reduced power consumption directly results in longer operating time for the product, which is a very desirable characteristic. Even for non-battery-powered products, reduced power consumption brings many advantages, such as reduced cost because of cheaper packaging or higher performance because of lower temperatures. Finally, reduced power consumption often leads to lower system cost (no fans required; no or cheaper air conditioning for data / telecom center etc.). Dynamic power consumption is caused by the charging and discharging of capacitances when a circuit switches. In addition, during switching a short-circuit current flows, but this current is typically much smaller, and will therefore be neglected in the following. The dynamic current due to capacitance charging and discharging is determined by the following well- known relationship: Pdyn ~ f • CL • Vdd 2
  • 3 Based on constant electrical field scaling, Vdd and CL each are reduced by30% in each successive process generation. Also, delay decreases by 30%,resulting in 43% increase in frequency. Therefore, the dynamic powerconsumption per device is reduced by 50% from one process generation tothe next. As scaling also doubles the number of devices that can beimplemented in a given die area, dynamic power consumption per areashould stay roughly identical. However, historically frequency has increasedby significantly more than 43% from one process generation to the next (e.g.in microprocessors, it has roughly doubled, due to architecturaloptimizations, such as deeper pipeline stages), and in addition, die sizes haveincreased with each new process technology, further increasing the powerconsumption, due to an increased number of active devices [5]. For thesereasons, dynamic power consumption has increased exponentially, as isshown in Figure 1-1 for the example of microprocessors. Reduction of leakage power consumption today is primarily a concernfor products that are powered by battery and spend most of their operatinghours in some type of standby mode, such as cell phones. For many process generations, however, leakage has increased roughlyby a factor of 10 for every two process nodes [6]. Due to this dramaticincrease with newer process generations, leakage is becoming a significantcontribution to overall IC power consumption even in normal operatingmode, as can be seen in Figure 1-1 as well. Leakage was estimated toincrease from 0.01% of overall power consumption in a 1.0µm technology,to 10% in a 0.1µm technology [6]. For a microprocessor, Intel estimatedleakage power consumption at more than 50W for a 100nm technologynode[3]. This figure probably is extreme, and leakage depends strongly on anumber of factors, such as threshold voltage (Vth) of the transistor, gateoxide thickness and environmental operating conditions (supply voltage Vdd,temperature T). Nevertheless, for an increasing number of products leakagepower consumption is turning into a problem, even when they are notbattery-powered.
  • 4 Figure 1-1. Development of dynamic and leakage power consumption over time [3,7] 1.3 TRADITIONAL APPROACHES TO POWER REDUCTION As outlined above, dynamic power consumption is governed by: Pdyn ~ f • CL • Vdd 2 with f denoting the switching frequency, CL the capacitance being switched, and Vdd the supply voltage . This formula immediately identifies the key levers to reduce dynamic power: • Reduce operating frequency • Reduce driven capacity • Reduce supply voltage Traditionally, reduction in supply voltage Vdd has been the most often followed strategy to reduce power consumption. Unfortunately, lowering Vdd has the side effect of reducing performance as well, primarily because gate
  • 5overdrive (the difference between Vdd and Vth) diminishes if the thresholdvoltage Vth is kept constant. Based on the alpha power law model [8], thedelay td of an inverter is given by CL • Vdd td = (Vdd − Vth )α with α denoting a fitting constant. As supply voltages are driven below1.0V, the reductions in gate overdrive are more pronounced than previously.In addition, newer process technologies give significantly less of aperformance boost compared to the previous process generation than hastraditionally been the case, therefore a further reduction in performance ishighly undesirable. Finally, the power reduction achieved by moving to anew process generation has trended down over time, since supply voltageshave been scaled by increasingly less than the 30% prescribed by theconstant electrical field scaling paradigm. Consequently, more advanced approaches are required. In the following, our main focus will be on dynamic power consumption,but we will also consider leakage power consumption.1.4 ZERO-VTH DEVICES The concept of zero-Vth devices was developed in the mid 90-ies. Itovercomes the diminishing gate overdrive by radically setting the thresholdvoltage of the active devices to zero. It has been shown [9], that the optimumpower dissipation is obtained, if Pleak (leakage contribution) is in the sameorder of magnitude as Pdyn (dynamic switching contribution). This can beachieved for transistors with Vth close to 0V (‘zero-Vth transistor‘). Thereforethe devices will never completely switch off. But from an overall powerperspective the gain in active power consumption is tremendous. Using these transistors the supply voltage of 130nm circuits can bereduced to values below 0.3V to achieve a Pdyn reduction by 90% withoutperformance degradation. Alternatively, the circuit can be operated at twicethe clock frequency when keeping the supply voltage at 1.2V, as shown inFigure 1-2. The corresponding Ion/Ioff-ratio for the zero-Vth transistor is about10-100 instead of >105 for the standard transistor options. During standby,the complete circuits are switched-off or are set into a low leakage mode tocope with the very high leakage contribution. The low leakage mode isachieved by ‘active well’ control, which denotes the use of the body effect.The well potentials of the PFETs and NFETs are altered to change Vth. Toachieve a lower leakage current, the absolute value of Vth is increased by
  • 6 reverse back biasing: a negative well-to-source voltage Usb is used. Therefore voltages below Vss for NFETs and above Vdd for PFETs have to be generated. Furthermore, active well is required to compensate the lot-to- lot or wafer-to-wafer variations of Vth. The initial ‘zero-Vth’ concept assumed constant junction temperatures Tj below 40°C. For some high-end computer equipment the costs for active chip cooling are affordable to achieve this junction temperature. But this is definitely not the case for cost-driven consumer products. For this application domain Tj in active mode ranges between 85°C and 125°C, and in some applications the specified worst-case ambient temperature is even 80°C. The proposed zero-Vth concept is therefore not applicable without changes and adaptations. Figure 1-2. Simulated performance curves of transistors with ultra-low Vth. Compared to low- Vth, either a performance gain or a Vdd reduction can be achieved. Curves for reg-Vth and high-Vth transistors of a 130 nm technology are included A more conservative approach with respect to zero-Vth, but still aggressive compared to current devices, had to be chosen. An ultra-low Vth device with about 150mV threshold voltage proved to be the best
  • 7compromise between zero-Vth and current low-Vth of about 300mV within a130 nm CMOS technology. To identify the optimal choice of Vth and Vdd in combination with thehigher junction temperature Tj, simulations with modified parameters of the130nm low-Vth transistor are performed. In Figure 1-3 the power dissipationis shown for a high activity circuit ( = 20%) with various options for thetransistor threshold voltages: reg-Vth, low-Vth, and transistors whose Vth arereduced to 200mV, 150mV, 100mV and 50mV. The reg-Vth circuitperformance was used as the reference (Vdd = 1.5V), and the supply voltagesfor the other transistor options were reduced to meet that referenceperformance. 3,5E-05 V dd= 1.5V T= 125°C fast 3,0E-05 Power [W ] 2,5E-05 nom = target 2,0E-05 1.2V slow 1,5E-05 1.0V 0.6V 1,0E-05 0.8V 0.7V 5,0E-06 0,0E+00 reg-Vt low-Vt 200mV 150mV 100mV 50mV Device Option / Vth (mV) Figure 1-3. Power dissipation at T=125°C in active mode for several transistor options with reduced Vth. A minimum power consumption is achieved at 150mV Vth. (At T=55°C the minimum is achieved for the same option but process variations show less impact). The reduced supply voltage leads to lower overall active powerconsumption Pactive. A minimum power consumption is reached at Vth =150mV. With even lower threshold voltages Pactive starts to increase againbecause of the increase of the leakage current. The steep rise of Pactiveoriginates from the exponential relation between Vth and leakage current. Asa rule of thumb a 100mV reduction of the threshold voltage allows for a Vdd
  • 8 reduction by 0.15V but on the other hand results in a tenfold increase of the leakage current. From Figure 1-3 also the impact of technology variations is visible. Due to the high leakage contribution a power reduction of only 25% is achieved under fast process conditions. Using back biasing in reverse mode, the high performance of fast transistors can be reduced through increasing Vth. The corresponding leakage current therefore decreases and allows a power reduction by 50% (stippled arrow). A process modification has been developed to manufacture devices with the threshold voltage of 150 mV, which proves to be the most efficient for the target application domain of mobile consumer products [10]. In Table1-1 the key transistor parameters of our ultra-low-Vth FETs (ulv) and of the standard low-Vth transistor are listed. The Vth values are 165mV and 161mV for the ulv-NFET and ulv-PFET respectively, Ion increases by 35% and 22%, which translates into an average decrease of the CV/I-metric delay by 29%. Circuit simulations showed a performance increase of 25%. Concerning Vth, performance, and Ioff the target values have been nearly met. Table 1-1. Extracted key parameters of the ulv-FETSs in comparison with the target values and the low- Vth FETs 130nm low-Vt 130nm ulv-FET Target NFET / PFET NFET / PFET Ion 560 / 240 755 / 295 [µA/µm] Ioff 1.2 / 1.2 48 / 17 35 [nA/µm] Vth 295 / 260 165/160 150 [mV] body effect 150 / 135 60/65 90 [mV/V] Vth@ L=10nm 35 / 30 65/30 [mV] Vth@ L=15nm 65 / 70 100/90 [mV] Simulated gate delay 1 0.8 0.75 [relative units] The sensitivity of Vth to gate length variation (roll-off) is expressed in Vth-shift per 10nm or 15nm gate length decrease. A comparison with low- Vth-FETs shows a pronounced increase. Therefore in addition to temperature compensation, back biasing has also to be used to compensate for this strong technology variation.
  • 9 The values of the body effect are also included in Table 1-1. The bodyeffect is expressed in Vth-shift per 1V well bias. The ulv-FETs yield values,which are lower by more than 50% compared to the low-Vth transistors. Thedecrease of body effect in combination with the increased roll-off reducesthe leverage of back biasing for ulv-FETs very significantly. The leverage isnot even sufficient to compensate the technology variation, since the valueof the roll-off is higher than that of the body effect. As an example, the ulv-NFET shows roll-off values of 65mV/10nm and 100mV/15nm and a bodyeffect of only 60mV/V. To investigate the migration potential of the ulv-FETs for futuretechnology generations Ioff measurement results, obtained from a recent90nm hardware, were used. Based on this measurement data the leverage ofactive well with the standard reg-Vth and low leakage transistor options hasbeen analyzed. For supply voltages of 1.2V and 0.75V a reverse back biasingvoltage of 0.5V has been applied. For the NFET, the back biasing results in aleakage reduction by 50% to 70% for all transistor widths and for bothvalues of Vdd. In the case of the PFET, the leakage reduction values aresimilar (60% to 80%) for transistors with W> 0.5µm. For very narrowPFETs with Vdd = 1.2V, the reduction is only 20% or even less. Sincenarrow FETs are used within SRAMs, which contribute a major part of thecircuit’s standby current, this small reduction for narrow transistors inaddition reduces significantly the leverage of active well. The root cause isan additional leakage mechanism based on tunnelling currents across thedrain-well junction, which limits the reverse back biasing to 0.5V. Thistunnelling current depends exponentially on the drain-well voltage and isworking against any reduction of the sub-threshold current via active well.At Vdd = 0.75V the drain-well voltage is reduced and the tunnelling currentis therefore lower. In this case the effect of back biasing is not compensatedby a rising tunnelling current and a leakage current reduction by 70% is stillachieved. For a 90nm technology the limit of 0.5V for the well potential swinglimits the reduction of the leakage currents to a factor between 2 and 4. Thisis still a major contribution of all feasible measures to reduce standby powerconsumption, but the leverage becomes quite small compared to thereduction ratios of several orders of magnitude obtained in previoustechnologies [11,12]. In future technologies, Ileak will become more stronglyaffected by the emerging tunnelling current Igate through the gate of the FET.This is due to the ever decreasing gate oxide thickness and also due to thefact, that even the on-state transistors shows gate leakage. Igate is not affectedby well biasing reducing the leverage of active well even further.
  • 10 In summary the zero-Vth-devices have become very susceptible to process and temperature variations. Significant yield is only achievable with back biasing via active well control and with active cooling. The latter approach is not feasible for mobile applications. Therefore a more conservative approach with respect to zero-Vth, but still aggressive compared to current devices, had to be chosen. An ultra-low-Vth device with about 150mV threshold voltage proved to be the best compromise between zero- Vth and current low-Vth of about 300mV within a 130 nm CMOS technology. But even though fabrication of this ultra-low-Vth device is possible, it affects some standard methods to overcome short-channel effects. The so called halo- or pocket-implantation had to be removed to bring the threshold voltage down. Unfortunately short-channel effects are now heavily increased, leading as shown to a very strong Vth roll-off at slight variations of the channel length. Finally this effect was prohibitive for the overall approach and led to cancellation of many zero-Vth projects in the industry[13]. 1.5 DESIGN APPROACHES TO POWER REDUCTION As outlined above, solutions from process technology by itself will not suffice to provide sufficient power reduction. Therefore, solutions must be found in algorithms, product architecture and logic design. Increasingly, differentiated device options provided by process technology are utilized on these levels in the search for optimization of power consumption. For leading-edge products which need to optimize both power consumption and system performance, optimization techniques on architecture and design level have been proposed and partly already been implemented. While academic research often focuses on the tradeoff between power consumption and performance, industrial product development must also take other variables into consideration. • Product cost: often, power optimization design techniques increase die area, directly affecting manufacturing cost. Also, utilization of additional devices (e.g. different Vth devices) increases mask count and consequently manufacturing cost, and additionally requires up-front expenditures for the development of such devices. Finally, increased manufacturing complexity poses the risk of lowered manufacturing yield. • Product robustness: it must be ensured that optimized products still work across the specified range of operating conditions, also taking manufacturing variations into account.
  • 111.5.1 Multi-Vdd Design As outlined in the introduction, the supply voltage Vdd quadraticallyimpacts dynamic switching power consumption. Thus, lowering Vdd is thepreferred option to reduce dynamic power consumption. However, asdiscussed in Section 1.2, lowering Vdd reduces the system performance.Thus, the incentive to lower Vdd to reduce power consumption is kept incheck by the need to maintain performance. Reduction of Vdd can be applied on different abstraction levels of adesign. Most effective regarding power reduction, and also easiest toimplement is to lower Vdd for an entire IC. As this will directly impact theperformance of the IC design, this often is not an option. On a lowerabstraction level, it is possible to lower Vdd for an entire module. This is stillrather simple to implement, but if only modules are chosen such that overallIC performance is not impacted, the achieved gains in power reduction willoften be very moderate. Finally, a reduction in supply voltage can be applied specifically toindividual gates, such that the overall system performance is not reduced.This approach, as shown in Figure 1-4, recognizes that in a typical design,most logic paths are not critical. They can be slowed down, oftensignificantly, without reducing the overall system performance. This slowingdown is achieved by lowering the supply voltage Vdd for gates on the non-critical paths, which results in lowered power consumption.
  • 12 10ns SET SET D Q D Q CLR Q CLR Q SET SET D Q D Q CLR Q CLR Q 5ns Non-critical path may be delayed 10ns SET SET D Q D Q Vdd_low Vdd_low CLR Q CLR Q Vdd_low SET SET D Q D Q CLR Q CLR Q 8ns Non-critical path runs with reduced supply voltage Figure 1-4. Multi-Vdd design This technique will modify the distribution of path delays in a design to a distribution skewed towards paths with higher delay, as indicated Figure 1-5 [14]. Single Supply Voltage SSV Multiple Supply Voltages MSV MSV SSV crit. paths td td 1/f 1/f Figure 1-5. Distribution of path delays under single and multiple supply voltages
  • 13 A number of studies have shown significant variation in dynamic powerreduction results from implementing a multi-Vdd design strategy, rangingfrom less than 10% up to almost 50%, with 40% being the average [15,16].Rules of thumb for selecting appropriate supply voltage levels have beendeveloped. When using two supply voltages, the lower Vdd was proposed tobe 0.6x-0.7x of the higher Vdd [17]. The optimal supply voltage level alsodepends on Vth [18]. The benefit of using multiple supply voltages quickly saturates. Themajor gain is obtained by moving from a single Vdd to dual-Vdd. Extendingthis to ever more supply voltage levels yields only small incremental benefits[18,19], even when the overhead introduced by multiple supply voltages (seebelow) is not taken into consideration. The power reduction achieved by this technique roughly depends on twoparameters: the difference between the regular supply voltage Vdd and thelowered supply voltage Vdd_low, and the percentage of gates to which Vdd_lowis applied. Regarding the first parameter, it has been pointed out some years ago thatthe leverage of this concept decreases as process technologies are scaleddown further [18]. Recent work has analyzed this in more detail [14]. At least for high-Vthdevices, which are essential for low standby power design due to their lowerleakage current, Vth has scaled much slower than Vdd recently. Therefore,gate overdrive (Vdd - Vth) is diminished, negatively impacting performance.Thus, even a little reduction in Vdd will have a very significant impact onperformance. Therefore, the potential to lower Vdd while maintaining overallsystem performance is greatly reduced. It is shown that from 0.25µm downto 0.09µm, the effectiveness of dual-Vdd decreases by a factor of 2 (from60% dynamic power reduction to 30%) for high-Vth designs, whereas it staysabout constant for low-Vth designs. This can however be countered byintroduction of variable threshold voltages, as will be seen later. Regarding the second parameter, experience has shown that especially indesigns using the multi-Vth technique outlined below, path delays tend to beskewed to higher delays already, thus reducing the number of gates that canbe slowed down further [14]. For the selection of those gates which will receive the lower supplyvoltage Vdd_low, a number of techniques have been proposed. Most prevalentis the concept of clustered voltage scaling (CVS). It recognizes that it isdesirable to have clusters of gates assigned to the same voltage, sincebetween the output of a gate supplied by Vdd_low and the input of a gatesupplied by Vdd a level shifter is required to avoid static current flow [20]. This concept has been enhanced by extended clustered voltage scaling(ECVS)[17] which essentially allows an arbitrary assignment of supply
  • 14 voltage levels to gates. This strategy implies more frequent insertion of level shifters into the design. However, usually only power consumption and delay are considered in the literature. The additional area cost is neglected. In industry, this certainly is not feasible. While conceptually simple, the implementation of a multi-Vdd concept poses a number of challenges. • The additional supply voltage Vdd_low needs to be created on-chip by a dc- to-dc converter, unless the voltage already exists externally. This results in area overhead, and in power consumption for the converter. • The additional supply voltage Vdd_low must be distributed across the chip. • Level-shifters are required between different supply domains. It is feasible to integrate level shifters into flip-flops [21]. The penalties in area, power consumption and delay resulting from these effects are not always taken into account by work published in the literature. Studies indicate that a 10% area overhead will result from implementing a dual-Vdd design [22]. An additional consideration for industrial IC product development is that EDA tool support for implementing a dual-Vdd design is still only rudimentary. It is not sufficient to have a single point tool which can perform power-performance tradeoffs. Instead, this methodology needs to encompass the entire design flow (e.g. power distribution in layout; automated insertion of level shifters etc.). 1.5.2 Multi-Vth Design Another essential technique is the use of different transistor threshold voltages (multi-Vth design). Primarily this technique reduces leakage power consumption, thus increasing standby time of battery-powered ICs. As leakage power consumption becomes an increasingly important component of overall power consumption in modern process technologies, this technique increasingly also helps to reduce overall power consumption significantly, as design moves to more advanced process technologies. The idea is similar to multi-Vdd design: paths that do not need highest performance are implemented with special leakage-reduced transistors (typically higher Vth transistors, but also thicker gate-oxide Tox), as shown in Figure 1-6.
  • 15 10ns SET SET D Q D Q CLR Q CLR Q SET SET D Q D Q CLR Q CLR Q 5ns Non-critical path may be delayed 10ns SET SET D Q D Q high Vt high V Q t Q CLR high Vt CLR SET SET D Q D Q CLR Q CLR Q 8ns Non-critical path runs with increased threshold voltage Figure 1-6. Multi-Vth design A typical industrial approach today is to first create a design using lowerVth transistors to achieve the required performance and then to selectivelyreplace gates off the critical path with higher Vth (or thicker Tox) transistorsto reduce leakage. Studies in the literature have reported reductions in leakage of around50% up to 80%. Some approaches assume that different Vth levels areprovided by the process technology (through doping variations) and proposealgorithms to optimally assign Vth levels to transistors, ensuring thatperformance is not compromised [23, 24]. Recently, it has also beenproposed to achieve modifications in Vth by modifying transistor length orgate oxide thickness Tox [25]. Design-tool support for this technique is also rudimentary at best. Whileit is becoming established to design different modules of an IC with differentVth transistors, it is very challenging to do this on the level of individualtransistors within a module. The primary reason is that the entire design flowmust be able to handle cells with identical functionality and size, whichdiffer in their electrical properties. This poses no principal algorithmicproblems, but must be consistently implemented in all EDA tools within adesign flow.
  • 16 1.5.3 Hybrid Approaches Recently approaches have been suggested in the literature which combine implementation of multiple supply voltages and multiple threshold voltages for further power reduction. Especially for designs where minimization of total power consumption is key (as compared to e.g. minimization of standby power for mobile products), it is possible to trade off leakage and dynamic power, as originally proposed in the zero-Vth concept. Studies in the literature indicate a total power optimum when leakage power contributes 10% to 30% [26,12]. This ratio depends significantly on the process technology, operating environment, and clock frequency of a design. For applications where leakage power minimization is critical (e.g. mobile products), this approach usually is not feasible, as it requires a relatively low Vth which causes high leakage currents [14]. With the increasing significance of gate leakage currents, variations of gate oxide thickness Tox have also been proposed. An overall framework for using two supply voltages and two threshold voltages as well has been presented [19]. Theoretically, it is shown that more than 60% of total power consumption can be saved this way (not considering required overhead such as level shifters, routing etc.). Rules of thumb are proposed and it is shown that the optimal second Vdd is about 50% of the original Vdd in this case. It is also argued that the usefulness of multi- Vdd strategies is not diminished, but actually increased in more advanced technologies, if also a multi-Vth strategy is followed, since this strategy allows to trade off leakage vs. dynamic power consumption by changing Vth and Vdd to optimize power consumption, while maintaining a required timing performance. This approach has been applied to the practical example of an ARM processor in [27]. Due to specific layout considerations it was not possible to implement all four intended combinations of Vdd and Vth. Instead, three different libraries were implemented. Using a CVS algorithm, a reduction in dynamic power by 15% was achieved for a 0.18µm process technology. Leakage power was reduced by 40%. As leakage power was more than 1000x smaller than dynamic power, overall active power reduction was 15%. To achieve this, a 14% increase in area was required. A very recent approach considers also transistor width sizing in addition to Vdd and Vth assignment [28]. Using a two stage, sensitivity-based approach, total power savings of 37% on average over a suite of benchmark circuits are reported. In this study, the threshold voltage is chosen rather low, so that leakage represents 20-50% of total power consumption. Therefore, optimization of both leakage and dynamic power consumption is essential, which is achieved with the presented approach.
  • 17 An enhanced approach for leakage power consumption considersmultiple gate oxide thicknesses Tox in addition to multi-Vth [29]. It ismotivated by the fact that gate leakage increases very dramatically withnewer process technologies. Gate leakage is of the same order of magnitudeas subthreshold leakage at the 90nm process node. Their relationship alsodepends significantly on the operating temperature T. The key observationthat an OFF transistor suffers from subthreshold leakage, an ON transistorfrom gate leakage, motivates the approach to analyze transistor states instandby mode and assign Vth and Tox such that leakage power consumptionis minimized. Leakage reductions of 5-6x are obtained on benchmarkcircuits, compared to designs using a single Vth and Tox. Previous approaches that included Tox into the optimization varied Toxonly for different design modules, not on critical paths within modules. These newer approaches promise further reductions in powerconsumption. This will come, however, at a price (as seen e.g. in the ARMexample). Design complexity increases significantly when variations inmany parameters are made available at the same time. In some studies, theresulting overhead is not considered.1.5.4 Cost Tradeoffs This overhead must be considered, however, since it is quite significant:• Multi-Vdd: level-shifter (area, power consumption, delay), routing of additional supply voltages (area).• Multi-Vth: additional masks (manufacturing costs); potentially special design rules at the boundary between different Vth devices (area).• Multi-Tox: additional masks (manufacturing costs).• In addition, IC development costs increase due to more complex design flows. Also, special process options (Vth, Tox) must be developed, qualified and continuously monitored. For each such option, the design library must be electrically characterized, modelled for all EDA tools, and potentially optimized regarding circuit design and layout. It must be maintained and regularly updated (changes in electrical parameters, changes in tools in the design flow) over a long period of time as well. If a very specialized manufacturing flow is developed to fully optimize a given product, it will be very difficult to shift manufacturing of this product to a different fab (e.g. a foundry in case additional capacity is required). For these and potentially other reasons, we are not yet aware of industrialproducts that have implemented such proposals in a fine-grained manner (i.e.different Vth, Vdd and Tox combined within one design module).
  • 18 Some approaches in the literature also determine optimum levels of threshold voltages depending on a given design. In industry, this is rarely feasible. Typically, a manufacturing process has to be taken as given, with only predefined values of Vth (and Tox) being available. 1.6 APPROACHES ON HIGHER ABSTRACTION LEVELS The approaches outlined above on gate level and device level can be (and often must be) supported by measures on higher levels of abstraction. Some of the most promising concepts are as follows: • partitioning the system such that large areas can be powered off for significant periods of time (block turnoff) • especially partitioning memory systems such that large parts can be turned off in standby mode • clock gating is an essential method which reduces dynamic power consumption by local off-switching of non-active gates • coding strategies (e.g. for buses) can reduce switching and thus dynamic power consumption 1.7 CONCLUSION AND FUTURE CHALLENGES There is no single “silver bullet” to solve the challenge of power reduction. While ultra-low voltage logic based on special ultra-low-Vth devices is a conceptually very convincing concept, its widespread implementation is hindered by manufacturing concerns. An extrapolation of current technology trends indicates that such a concept will become even more difficult in the future. Today, design techniques are the most promising approach to reduce power – both dynamic and leakage. The concepts outlined here can be further extended. It is feasible to dynamically adjust supply and threshold voltages. These are theoretically promising concepts which however still require more investigation especially with regard to feasibility under industrial boundary conditions. Quite likely, in the future even more emphasis than today will have to be placed on power reduction schemes on algorithmic and system level. On these levels, the levers to reduce power consumption are largest. Acknowledgement The authors wish to acknowledge and thank Jörg Berthold and Tim Schönauer for their contributions and fruitful discussions.
  • 19 References[1] G. Moore, Cramming More Components onto integrated circuits, Electronics Magazine, Vol. 38, No. 8, 1965, pp. 114-117.[2] ITRS, International Technology Roadmap for Semiconductors, 2003, http://public.itrs.net.[3] F. Pollack, New Microarchitecture Challenges in the Coming Generations of CMOS Process Technologies, Micro32 Keynote, 1999.[4] U. Schlichtmann, Systems are Made from Transistors: UDSM Technology Creates New Challenges for Library and IC Development, IEEE Euromicro Symposium on Digital System Design, 2002, pp. 1-2.[5] S. Borkar, Design Challenges of Technology Scaling, IEEE Micro, July/August 1999, pp. 23-29.[6] S. Thompson, P. Packan, and M. Bohr, MOS Scaling: Transistor Challenges for the 21st Century, Intel Technology Journal, Q3 1998.[7] N. Kim et al., Leakage Current: Moores Law Meets Static Power, IEEE Computer, Vol. 36, No. 12, December 2003, pp. 68-75.[8] S. Sakurai, A. R. Newton, Alpha-Power Law MOSFET Model and its Application to CMOS Inverter Delay and Other Formulas, IEEE Journal of Solid-State Circuits, Vol. 25, No. 2, 1990, pp. 584-594.[9] J.B. Burr, J. Schott, A 200 mV self-testing encoder/decoder using Stanford ultra-low- power CMOS, 1994 IEEE International Solid-State Circuits Conference[10] J. Berthold, R. Nadal, C. Heer, Optionen für Low-Power-Konzepte in den sub-180-nm- CMOS-Technologien (In German), U.R.S.I. Kleinheubacher Tagung 2002.[11] V. Svilan, M. Matsui, J. B. Burr, Energy-Efficient 32 x 32-bit Multiplier in Tunable Near-Zero Threshold CMOS, ISLPED 2000, pp. 268-272.[12] V. Svilan, J. B. Burr, L. Tyler, Effects of Elevated Temperature on Tunable Near-Zero Threshold CMOS, ISLPED 2001, pp. 255-258.[13] C. Heer, Designing low-power circuits: an industrial point of view, PATMOS 2001[14] T. Schoenauer, J. Berthold, C. Heer, Reduced Leverage of Dual Supply Voltages in Ultra Deep Submicron Technologies, International Workshop on Power And Timing Modeling, Optimization and Simulation PATMOS 2003, pp. 41-50.[15] K. Usami, M. Igarashi, Low-Power Design Methodology and Applications utilizing Dual Supply Voltages, Proceedings of the Asia and South Pacific Design Automation Conference 2000, pp. 123-128.[16] M. Donno, L. Macchiarulo, A. Macii, E. Macii, M. Poncino, Enhanced Clustered Voltage Scaling for Low Power, Proceedings of the 12th ACM Great Lakes Symposium on VLSI, 2002, pp. 18-23.[17] K. Usami et al., Automated Low-Power Technique Exploiting Multiple Supply Voltages Applied to a Media Processor, IEEE Journal of Solid-State Circuits, Vol. 33, No. 3, March 1998, pp. 463-472.[18] M. Hamada, Y. Ootaguro, T. Kuroda, Utilizing Surplus Timing for Power Reduction, Proceedings IEEE Custom Integrated Circuits Conference CICC, 2001, pp. 89-92.[19] A. Srivastava, D. Sylvester, Minimizing Total Power by Simultaneous Vdd/Vth Assignment, Proceedings of the Asia and South Pacific Design Automation Conference 2003, pp. 400-403.[20] K. Usami, M. Horowitz, Clustered Voltage Scaling Technique for Low-Power Design, Proceedings of the International Symposium on Low Power Design ISLPD, 1995, pp. 3- 8.
  • 20 [21] K. Usami et al., Design Methodology of Ultra Low-power MPEG4 Codec Core Exploiting Voltage Scaling Techniques, Proceedings of the 35th Design Automation Conference 1998, pp. 483-488. [22] C. Yeh, Y.-S. Kang, Layout Techniques Supporting the Use of Dual Supply Voltages for Cell-Based Designs, Proceedings of the 36th Design Automation Conference 1999, pp. 62-67. [23] Q. Wang, S. Vrudhula, Algorithms for Minimizing Standby Power in Deep Submicrometer, Dual-Vt CMOS Circuits, IEEE Transactions on CAD, Vol. 21, No. 3, March 2002, pp. 306/318. [24] L. Wei, Z. Chen, K. Roy, M. Johnson, Y. Ye, V. De, Design and Optimization of Dual- Threshold Circuits for Low-Voltage Low-Power Applications, IEEE Transactions on Very Large Scale Integration (VLSI), Vol. 7, No. 1, March 1999, pp. 16-24. [25] N. Sirisantana, K. Roy, Low-Power Design Using Multiple Channel Lengths and Oxide Thicknesses, IEEE Design & Test of Computers, January-February 2004, pp. 56-63. [26] K. Nose, T. Sakurai, Optimization of VDD and VTH for Low-Power and High-Speed Applications, Proceedings of the Asia and South Pacific Design Automation Conference 2000, pp. 469-474. [27] R. Bai, S. Kulkarni, W. Kwong, A. Srivastava, D. Sylvester, D. Blaauw, An Implementation of a 32-bit ARM Processor Using Dual Power Supplies and Dual Threshold Voltages, IEEE International Symposium on VLSI, 2003, pp. 149-154. [28] A. Srivastava, D. Sylvester, D. Blaauw, Concurrent Sizing, Vdd and Vth Assignment for Low-Power Design, Proceedings of the Design, Automation and Test in Europe Conference DATE, 2003, pp. 718-719. [29] D. Lee, H. Deogun, D. Blaauw, D. Sylvester, Simultaneous State, Vt and Tox Assignment for Total Standby Power Minimization, Proceedings of the Design, Automation and Test in Europe Conference DATE, 2003, pp. 494-499.
  • 21Chapter 2ON-CHIP OPTICAL INTERCONNECT FORLOW-POWERIan O’Connor and Fr´ d´ ric Gaffiot e eEcole Centrale de LyonAbstract It is an accepted fact that process scaling and operating frequency both contribute to increasing integrated circuit power dissipation due to interconnect. Extrapolat- ing this trend leads to a red brick wall which only radically different interconnect architectures and/or technologies will be able to overcome. The aim of this chap- ter is to explain how, by exploiting recent advances in integrated optical devices, optical interconnect within systems on chip can be realised. We describe our vision for heterogeneous integration of a photonic “above-IC" communication layer. Two applications are detailed: clock distribution and data communication using wavelength division multiplexing. For the first application, a design method will be described, enabling quantitative comparisons with electrical clock trees. For the second, more long-term, application, our views will be given on the use of various photonic devices to realize a network on chip that is reconfigurable in terms of the wavelength used.Keywords: Interconnect technology, optical interconnect, optical network on chip2.1 INTRODUCTION In the 2003 edition of the ITRS roadmap [17], the interconnect problem wassummarised thus: “For the long term, material innovation with traditional scal-ing will no longer satisfy performance requirements. Interconnect innovationwith optical, RF, or vertical integration ... will deliver the solution”. Continu-ally shrinking feature sizes, higher clock frequencies, and growth in complexityare all negative factors as far as switching charges on metallic interconnect isconcerned. Even with low resistance metals such as copper and low dielectricconstant materials, bandwidths for long interconnect will be insufficient for fu-ture operating frequencies. Already the use of metal tracks to transport a signalover a chip has a high cost in terms of power: clock distribution for instance
  • 22requires a significant part (30-50%) of total chip power in high-performancemicroprocessors. A promising approach to the interconnect problem is the use of an opticalinterconnect layer, which could empower an increase in the ratio between datarate and power dissipation. At the same time it would enable synchronous op-eration within the circuit and with other circuits, relax constraints on thermaldissipation and sensitivity, signal interference and distortion, and also free uprouting resources for complex systems. However, this comes at a price. Firstly,high-speed and low-power interface circuits are required, design of which isnot easy and has a direct influence on the overall performance of optical inter-connect. Another important constraint is the fact that all fabrication steps haveto be compatible with future IC technology and also that the additional costincurred remains affordable. Additionally, predictive design technology is re-quired to quantify the performance gain of optical interconnect solutions, whereinformation is scant and disparate concerning not only the optical technology,but also the CMOS technologies for which optics could be used (post-45nmnode). In section 2.2, we will describe the “above-IC” optical technology. Sections2.3 and 2.4 describe an optical clock distribution network and a quantitativeelectrical-optical power comparison respectively. A proposal for a novel opticalnetwork on chip in discussed in section OPTICAL INTERCONNECT TECHNOLOGY Various technological solutions may be proposed for integrating an opticaltransport layer in a standard CMOS system. In our opinion, the most promisingapproach makes use of hybrid (3D) integration of the optical layer above acomplete CMOS IC, as shown in fig. 2.1. The basic CMOS process remainsthe same, since the optical layer can be fabricated independently. The weaknessof this approach is in the complex electrical link between the CMOS interfacecircuits and the optical sources (via stack and advanced bonding). In the system shown in fig. 2.1, a CMOS source driver circuit modulatesthe current flowing through a biased III-V microsource through a via stackmaking the electrical connection between the CMOS devices and the opticallayer. III-V active devices are chosen in preference to Si-based optical devicesfor high-speed and high-wavelength operation. The microsource is coupled tothe passive waveguide structure, where silicon is used as the core and SiO2as the cladding material. Si/SiO2 structures are compatible with conventionalsilicon technology and silicon is an excellent material for transmitting wave-lengths above 1.2µm (mono-mode waveguiding with attenuation as low as 0.8dB/cm has been demonstrated [10]). The waveguide structure transports theoptical signal to a III-V photodetector (or possibly to several, as in the case of
  • 23 III−V III−V laser source photodetector electrical contact Si photonic waveguide (n=3.5) SiO2 waveguide metallic interconnect structure cladding (n=1.5) driver CMOS IC receiver circuit circuit Figure 2.1. Cross-section of hybridised interconnection structurea broadcast function) where it is converted to an electrical photocurrent, whichflows through another via stack to a CMOS receiver circuit which regeneratesthe digital output signal. This signal can then if necessary be distributed over asmall zone by a local electrical interconnect network.2.3 AN OPTICAL CLOCK DISTRIBUTION NETWORK In this section we present the structure of the optical clock distribution net-work, and detail the characteristics of each component part in the system: ac-tive optoelectronic devices (external VCSEL source and PIN detector), passivewaveguides, interface (driver and receiver) circuits. The latter represent ex-tremely critical parts to the operation of the overall link and require particularlycareful design. An optical clock distribution network, shown in fig. 2.2, requires a singlephotonic source coupled to a symmetrical waveguide structure routing to anumber of optical receivers. At the receivers the high-speed optical signal isconverted to an electrical one and provided to local electrical networks. Hencethe primary tree is optical, while the secondary tree is electrical. It is not feasibleto route the optical signal all the way down to the individual gate level sinceeach drop point requires a receiver circuit which consumes area and power.The clock signal is thus routed optically to a number of drop points which willcover a zone over which the last part of the clock distribution will be carried out
  • 24by the electrical secondary clock tree. The size of the zones is determined bycalculating the power required to continue in the optical domain and comparingit to the power required to distribute over the zone in the electrical domain. Thenumber of clock distribution points (64 in the figure) is a particularly crucialparameter in the overall system. The global optical H-tree was optimised to achieve minimal optical lossesby designing the bend radii to be as large as possible. For 20mm die width and64 output nodes in the H-tree at the 70nm technology mode, the smallest radiusof curvature (r3 in fig. 2.2) is 625µm, which leads to negligible pure bendingloss. die width, D L CR L CV : source−waveguide coupling loss LY LW : waveguide transmission loss LB : bending loss LY : Y−coupler loss L CR : waveguide−receiver r1 LB coupling loss r3 LW r2 L CV optical source electrical optical optical r1=D/8, r2=D/16, r3=D/32 clock trees waveguides receiversFigure 2.2. Optical H-tree clock distribution network (OCDN) with 64 output nodes. r1−3 arethe bend radii linked to the chip width D2.3.1 VCSEL sources VCSELs (Vertical Cavity Surface Emitting Lasers) are certainly the mostmature emitters for on-chip or chip-to-chip interconnections. Commercial VC-SELs, when forward biased at a voltage well above 1.5V, can emit opticalpower of the order of a few mW around 850nm, with an efficiency of some40%. Threshold currents are typically in the mA range. However, fundamentalrequirements for integrated semiconductor lasers in optical interconnect appli-cations are small size, low threshold lasing operation and single-mode operation(i.e. only one mode is allowed in the gain spectrum). Additionally, the factthat VCSELs emit light vertically makes coupling less easy. It is clear that
  • 25significant effort is required from the research community if VCSELs are tocompete seriously in the on-chip optical interconnect arena, to increase wave-length, efficiency and threshold current in the same device. Long wavelength,and low-threshold VCSELs are only just beginning to emerge (for example, a1.5µm, 2.5Gb/s tuneable VCSEL [5], and an 850nm, 70µA threshold current,2.6µm diameter CMOS compatible VCSEL [11] have been reported). Ulti-mately however, optical interconnect is more likely to make use of integratedmicrosources as described in section 2.5, as these devices are intrinsically bettersuited to this type of application.2.3.2 PIN photodetectors In order to optimise the frequency and power dissipation performance of theoverall link, photodetectors must exhibit high quantum efficiency, large intrinsicbandwidth and small parasitic capacitance. The photodetector performance ismeasured by the bandwidth efficiency product. Conventional III-V PIN devices suffer from two main limitations. On onehand, their relatively high capacitance per unit area leads to limitations in thedesign of the transconductance amplifier interface circuit. On the other hand,due to its vertical structure, there is a tradeoff between its frequency performanceand its efficiency (the quantum efficiency increases and the bandwidth decreaseswith the absorption intrinsic layer thickness) [9]. Metal-semiconductor-metal (MSM) photodetectors offer an alternative overconventional PIN photodetectors. An MSM photodetector consists of interdig-itated metal contacts on top of an absorption layer. Because of their lateralstructure, MSM photodetectors have very high bandwidths due to their lowcapacitance and the possibility to reduce the carrier transit time. However,the responsivity is usually low compared to PIN photodetectors [4]. MSMphotodiodes with bandwidth greater than 100GHz have been reported.2.3.3 Waveguides Optical waveguides are at the heart of the optical interconnect concept.In the Si/SiO2 approach, the high relative refractive index difference ∆ =(n2 − n2 )/2n2 between the core (n1 ≈ 3.5 for Si) and cladding (n2 ≈ 1.5 for 1 2 1SiO2 ) allows the realisation of a compact optical circuit with dimensions com-patible with DSM technologies. For example, it is possible to realise monomodewaveguides less than 1µm wide (waveguide width of 0.3µm for wavelengthsof 1.55µm), with bend radii of the order of a few µm [15]. However, the performance of the complete optical system depends on theminimum optical power required by the receiver and on the efficiency of passiveoptical devices used in the system. The total loss in any optical link is the sum
  • 26of losses (in decibels) of all optical components: Ltotal = LCV + LW + LB + LY + LCR (2.1)where LCV is the coupling coefficient between the photonic source and optical waveguide. There are currently several methods to couple the beam emitted from the laser into the optical waveguide. In this analysis we assumed 50% coupling efficiency LCV from the source to a single mode waveguide. LW is the rectangular waveguide transmission loss per unit distance of the optical power. Due to small waveguide dimensions and large in- dex change at the core/cladding interface in the Si/SiO2 waveguide the side-wall scattering is the dominant source of loss (fig. 2.3a). For the waveguide fabricated by Lee [10] with roughness of 2nm the calculated transmission loss is 1.3dB/cm. LB is the bending loss, highly dependent on the refractive index difference ∆ between the core and cladding medium. In Si/SiO2 waveguides, ∆ is relatively high and so due to this strong optical confinement, bend radii as small as a few µm may be realised. As can be seen from fig. 2.3b, the bending losses associated with a single mode strip waveguide are negligible if the radius of curvature is larger then 3µm. LY is the Y-coupler loss, and depends on the reflection and scattering attenuation into the propagation path and surrounding medium. For high index difference waveguides the losses for the Y-branch are significantly smaller than for low ∆ structures and the simulated losses are less then 0.2dB per split [14]. LCR is the coupling loss from the waveguide to the optical receiver. Using currently available materials and methods it is possible to achieve an almost 100% coupling efficiency from waveguide to optical receiver. In this analysis the coupling efficiency is assumed to be 87% (LCR = 0.6dB) [16].2.3.4 Interface circuits High-speed CMOS optoelectronic interface circuits are crucial building blocksto the optical interconnect approach. The electrical power dissipation of thelink is defined by these circuits, but it is the receiver circuit that poses the mostserious design challenges. The power dissipated by the source driver is mainlydetermined by the source bias current and is therefore device-dependent. Onthe receiver side however, most of the receiver power is due to the circuit, whileonly a small fraction is required for the photodetector device.
  • 27 60 100 1 50 Transmission loss (dB/cm) 0.01 Pure bending loss (dB) 40 0.0001 30 1e-06 1e-08 20 1e-10 10 1e-12 0 1e-14 1 2 3 4 5 6 7 8 9 10 11 12 2 3 4 5 6 7 8 9 Sidewall roughness (nm) Bend radius (um)Figure 2.3a. Simulated transmission loss Figure 2.3b. Simulated pure bending lossfor varying sidewall roughness in a for various bend radii in a 0.5µm× 0.2µm0.5µm× 0.2µm Si/SiO2 strip waveguide Si/SiO2 strip waveguide2.3.4.1 Driver circuits. Source driver circuits generally use a currentmodulation scheme for high-speed operation. The source always has to bebiased above its threshold current by a MOS current sink to eliminate turn-ondelays, which is why low-threshold sources are so important (figures of theorder of 40µA [7] have been reported). A switched current sink modulatesthe current flowing through the source, and consequently the output opticalpower injected into the waveguide. As with most current-mode circuits, highbandwidth can be achieved since the voltage over the source is held relativelyconstant and parasitic capacitances at this node have reduced influence on thespeed. Receiver circuits. A typical structure for a high-speed pho-toreceiver circuit consists of: a transimpedance amplifier (TIA) to convert thephotocurrent of a few µA into a voltage of a few mV; a comparator to gener-ate a rail-to-rail signal; and a data recovery circuit to eliminate jitter from therestored signal. Of these, the TIA is arguably the most critical component forhigh-speed performance, since it has to cope with a generally large photodiodecapacitance situated at its input. The basic transimpedance amplifier structure in a typical configuration isshown in fig. 2.4 [8]. The bandwidth/power ratio of this structure can be max-imised by using small-signal analysis and mapping of the individual componentvalues to a filter approximation of Butterworth type. It is then possible to develop a synthesis procedure which, from desiredtransimpedance performance criteria (gain Zg0 , bandwidth and pole qualityfactor Q) and operating conditions (photodiode and load capacitances, Cd andCl respectively) generates component values for the feedback resistance Rf andthe voltage amplifer (voltage gain Av and output resistance Ro ). Circuits withhigh Ro /Av ratio (≈ 1/ gm ) require the least quiescent current and area andthis quantity constitutes therefore an important figure of merit in design space
  • 28 Rf 1 1 + Av Ii ω0= −A R oC y M f (M x + Mm(1 + Mx )) v M f (M x + M m(1 + M x ))(1 + A v ) Cd Cl Q= Vo 1 + M x (1 + M f ) + MmM f (1 + A v ) Vdd Vi Cm M2 M3 Vo Z g0 = − ( R f − Ro /Av 1 + 1/Av ) M f = Rf / R o M i = Cx / C y Mm= Cm / C y M1 Ci Co Cx = Cd + Ci Cy = C l + Co Figure 2.4. CMOS transimpedance amplifier structureexploration (fig. 2.5a). To reach a sized transistor-level circuit, approximateequations for the small-signal characteristics and bias conditions of the circuitare sufficient to allow a first-cut sizing of the amplifier, which can then be fine-tuned by numerical or manual optimisation, using simulation for exact results.The complete process is described in [13]. Amplifier Ro/Av requirement Ci=500fF Cl=100fF 1THzohm Transimpedance amplifier characteristics against technology node Ro/Av Cd = 400fF, Cl = 150fF 300 250 100 200 Area / um2 150 Quiescent power / 100uW 100400 50350300250 10200150100 50 0 1 1 10000 3 3000 Bandwidth Transimpedance requirement gain 0.1 /GHz 10 1000 requirement 350 180 130 100 70 45 /ohms Technology node (nm)Figure 2.5a. TIA Ro /Av design Figure 2.5b. Evolution of TIA character-space with varying bandwidth and istics (power, area, noise) with technologytransimpedance gain requirements node Using this methodology with industrial transistor models for technologynodes from 350nm to 180nm and predictive BSIM3v3/BSIM4 models for tech-nology nodes from 130nm down to 45nm [3], we generated design parametersfor 1T HzΩ transimpedance amplifiers to evaluate the evolution in critical char-acteristics with technology node. Fig. 2.5b shows the results of transistor levelsimulation of fully generated photoreceiver circuits at each technology node.
  • 292.4 QUANTITATIVE POWER COMPARISON BETWEEN ELECTRICAL AND OPTICAL CLOCK DISTRIBUTION NETWORKS2.4.1 Design methodology In an optical link there are two main sources of electrical power dissipation:(i) power dissipated by the optical receiver(s) and (ii) energy needed by theoptical source(s) to provide the required optical output power. To estimate theelectrical power dissipated in the system we developed the methodology shownin fig. 2.6. BER specification photodiode characteristics (SNR requirement) (R,Cd,Idark ) minimum optical transimpedance power at receiver amplifier losses in passive minimum optical waveguide network power at source source emitter receiver efficiency power power electrical power dissipated in optical systemFigure 2.6. Methodology used to estimate the electrical power dissipation in an optical clockdistribution network The first criterion for defining the performance of the optoelectronic link isthe required signal transmission quality, represented by the bit error rate (BER)and directly linked to the photoreceiver signal to noise ratio. For an on-chipinterconnect network, a BER of 10−15 is acceptable. To calculate the requiredsignal power at the receiver, the characteristics of the receiver circuit have tobe extracted from the transistor-level schematic, which is generated from thephotodetector characteristics (responsivity R, Cd , dark current Idark ) and fromthe required operating frequency using the method described in section 2.3.For the given BER and for the noise signal associated with the photodiode andtransimpedance circuit the minimum optical power required by the receiverto operate at the given error probability can be calculated using the Morikuniformula [12]. With this figure, and knowing the layout and therefore the optical losses thatwill be incurred in the waveguides, the minimum required optical power at thesource can be estimated. The total electrical power dissipated in the optical
  • 30link is the sum of the power dissipated by the number of optical receivers andthe energy needed by the source to provide the required optical power. Theelectrical power dissipated by the receivers can be extracted from transistor-level simulations. To estimate the energy needed by the optical source, laserlight-current characteristics given by Amann [1] were used.2.4.2 Design performance Our aim in this work was to quantitatively compare the power dissipationin electrical and optical clock distribution networks for a number of cases, in-cluding technology node prediction. For both electrical and optical cases weused technology parameters from the ITRS roadmap (wire geometry, materialparameters). For transistor models we used predictive model parameters fromBerkeley (BSIM3V3 down to 70nm and BSIM4 down to 45nm). The powerdissipated in the electrical system can be attributed to the charging and discharg-ing of the wiring and load capacitance and to the static power dissipated by thebuffers. In order to calculate the power we used an internally developed simu-lator, which allows us to model and calculate the electrical parameters of clocknetworks for future technology nodes [18]. For optical performance predictionswe used existing technology characteristics while for the optoelectronic deviceswe took datasheets from two real devices and used these figures. The power dissipated in clock distribution networks was analysed in bothsystems at the 70nm technology node. Power dissipation figures for electricaland optical CDNs were calculated based on the system performance summarisedin tables 2.1a and 2.1b.Table 2.1a. Electrical CDN characteris- Table 2.1b. Optical CDN characteristicstics Optical system parameter Electrical system parameter Wavelength λ (nm) 1550 Technology (nm) 70 Waveguide core index (Si) 3.47 Vdd (V) 0.9 Waveguide cladding index (SiO2 ) 1.44 Tox (nm) 1.6 Waveguide thickness (µm) 0.2 Chip size (mm2 ) 400 Waveguide width (µm) 0.5 Global wire width (µm) 1 Transmission loss (dB/cm) 1.3 Metal resistivity (Ω-cm) 2.2 Loss per Y-junction (dB) 0.2 Dielectric constant 3 Input coupling coefficient (%) 50 Optimal segment length (mm) 1.7 Photodiode capacitance (fF) 100 Optimal buffer size (µm) 90 Photodiode responsivity (A/W) 0.95 What follows is the results of comparisons of the power dissipation in elec-trical and optical clock distribution networks. This was quantitatively carriedout for varying chip size, operating frequency, number of clock distributionpoints, technology node, and finally sidewall roughness. This latter perfor-
  • 31mance characteristic is the only non system-driven characteristic, but it givessome important design information to technology groups working on opticalinterconnect. Fig. 2.7a shows a power comparison where we vary square die size from 10 to37 mm width. This analysis was carried out for the 70nm node at a distributionfrequency of 5.6GHz (which is the clock frequency associated with this node)and 256 drop points. Electrical CDN power rises almost linearly with die size,which is understandable since the line lengths increase and therefore requiremore buffers to drive them. Optical CDN power rises much more slowly sinceall that is really changing is transmission loss and this has a quite minor effecton the overall power dissipation. When we vary clock frequency for constant chip width, fig. 2.7b we observea similar effect for the electrical CDN. Again, the number of buffers has toincrease since the segment lengths have to be reduced in order to attain the lowerRC time constants. For the optical CDN, what is changing is the receiver powerdissipation. The transimpedance amplifier requires a lower output resistance inorder to operate at higher frequencies and this translates to a higher bias current. In fig. 2.7c, we vary the number of drop points and see that both electricaland optical CDN power dissipation rises, but optical rises much faster thanelectrical. There are two reasons for this: firstly, every time the number of droppoints is doubled, so is the number of receivers and this accounts for a large partof the power dissipation; secondly, the number of splitters is doubled, whichin turn means that the power at emission also has to be doubled. These twofactors cause the optical power to catch up with the electrical power at around4000 drop points. Fig. 2.7e shows a comparison for varying technology node. Not only thetechnology is changing here, we are also changing the clock frequency asso-ciated with the node. We can see that at the 70nm node there is a five-folddifference between electrical and optical clock distribution. As the technologynode advances, this difference becomes even more marked. A final analysis, fig. 2.7f, shows how technological advances are requiredto improve system performance, concerning in this case waveguide sidewallroughness. 5nm roughness translates to a transmission loss of around 8dB/cm,which in turn corresponds to a power dissipation figure of around 500mWfor the 70nm node at 5.6GHz and 20mm chip width. Looking at the 2nmroughness point, achieved at MIT [10] and corresponding to a transmissionloss of 1.3dB/cm, we obtain a power dissipation figure of about 10mW, a fifty-fold decrease in the overall power dissipation by going from 5nm roughness to2nm roughness. This demonstrates the importance of optimising the passivewaveguide technology for the whole system.
  • 32 1600 1400 Electrical CDN Optical CDN Electrical CDN 256 1400 1200 Optical CDN 256 Electrical CDN 128 Power dissipation (mW) Power dissipation (mW) Optical CDN 128 1200 1000 1000 800 800 600 600 400 400 200 200 0 100 300 500 700 900 1 3 5 7 Die size (mm2) Clock frequency (GHz)Figure 2.7a. Comparison of power dissi- Figure 2.7b. Comparison of power dissi-pation in electrical and optical clock dis- pation in electrical and optical clock dis-tribution networks for varying chip size tribution networks for varying clock fre-(70nm technology, 5.6GHz, 256 drop quency (70nm technology, 400mm2 , 256points) drop points) 10000 30 Electrical CDN 20 Optical CDN Power gain optical/electrical (%) 1000 10 Power dissipation (mW) 0 100 -10 -20 10 -30 -40 1 -50 4 32 256 2048 8172 4 32 256 2048 8172 Number of drop points (nodes) Number of drop points (nodes)Figure 2.7c. Comparison of power dissi- Figure 2.7d. Comparison of power dissi-pation in electrical and optical clock dis- pation in electrical and optical clock dis-tribution networks for varying number of tribution networks for varying number ofdrop points (70nm technology, 5.6GHz, drop points (70nm technology, 5.6GHz,400mm2 ) 400mm2 ) 1200 Electrical CDN 1000 Optical CDN 256 Optical CDN Optical CDN 128 1000 Power dissipation (mW) Power dissipation (mW) 800 100 600 400 10 200 0 130 100 70 45 1 3 5 7 9 Technology node (nm) Waveguide transmission loss (dB/cm)Figure 2.7e. Comparison of power dissi- Figure 2.7f. Evaluation of power dissipa-pation in electrical and optical clock dis- tion in optical clock distribution networkstribution networks for varying technology for varying waveguide sidewall roughnessnodes (70nm technology, 5.6GHz, 400mm2 ) For a BER of 10−15 the minimal power required by the receiver is -22.3dBm(at 3GHz). Losses incurred by passive components for various nodes in theH-tree are summarised in table 2.2.
  • 33Table 2.2. Optical power budget for 20mm die width at 3GHzNumber of nodes in H-tree 16 32 64 128Loss in straight lines (dB) 1.3 1.3 1.3 1.3Loss in curved lines (dB) 1.53 1.66 1.78 1.85Loss in Y-dividers (dB) 12 15 18 21Loss in Y-couplers (dB) 0.8 1 1.2 1.4Output coupling loss (dB) 0.6 0.6 0.6 0.6Input coupling loss (dB) 3 3 3 3Total optical loss (dB) 19.2 22.5 25.8 29.1Min. receiver power (dBm) -22.3 -22.3 -22.3 -22.3Laser optical power (mW) 0.5 1.1 2.30 4.85 We can conclude from this analysis that power dissipation in optical clockdistribution networks is lower than that of electrical clock distribution networks,by a factor of five for example at the 70nm technology node. This factor will inthe future become larger due to two reasons: firstly due to improvements in opti-cal fabrication technology; and secondly with the rise in operating frequencies.However, this figure is probably not sufficient to convince semiconductor manu-facturers to introduce such large technological and methodological changes forthis application. To improve the figure, weak points can be identified for eachmain part of an integrated optical link. For the source, the efficiency betweenelectrical and optical power conversion is relatively low. This needs to be im-proved and one area is possibly in integrated microsources. For the waveguidestructures, most of the losses need to be improved, especially transmission lossand coupling loss. Sidewall roughness especially has a direct and considerableimpact on the power dissipation in the global system. Finally at the receiverend, the transimpedance amplifier power dissipation is too high. Better circuitstructures must be devised, or the photodetector parasitic capacitance needs tobe reduced.2.5 OPTICAL NETWORK ON CHIP In current SoC architectures, global data throughput between functionalblocks can reach up to tens of gigabits per second, the load being shared byseveral communication buses. In the future the constraints acting on such dataexchange networks will continue to increase: the number of IP blocks in anintegrated system could be as high as several hundred and the global throughputcould reach the Tb/s scale. To provide this level of performance, the communi-cation system itself is designed as an IP block into which the various functionalunits will be connected. This type of standardised hardware communicationarchitecture is called a network on a chip (NoC).
  • 34 Using wavelength division multiplexing (WDM) techniques, photonics andoptoelectronics may offer new solutions to realise reconfigurable optical net-works on chip (ONoC). An ONoC, as an electronic router with routing based onwavelength λ, is actually a circuit-switching based topology and can thus ensuredata exchanges between IP blocks with very low contention. The advantagesof using an optical network are many: independence of interconnect perfor-mance from distance and data rate, crosstalk reduction, connectivity increase,interconnect power dissipation reduction, increase in the size of isochronoustiles, use of communication protocols. Figure 2.8 shows a 4 × 4 ONoC with allelectronic interfaces: photodetector and laser in III-V technology and opticalnetwork in SOI technology, using similar heterogeneous integration techniquesas described in section 2.2. Intellectual property (IP) blocks shown can be pro-cessor cores, memory blocks, functional units etc. with standard interfaces tothe communication network. This is a multi-domain device with high speedoptoelectronic circuits (modulation of the laser current and photodetectors) andpassive optics (waveguides and passive filters). In the figure, M are masters(processor, IP, ...) which can communicate with targets T (memory, ...). Thenetwork is comprised of 4 stages, each associated with a single resonant wave-length. The operation of the 4×4 network is summarised in the table of figure2.3. This system is a fully passive circuit-switching network based on wave-length routing and is a non-blocking network. From Mi to Tj , there exists onlyone physical path associated with one wavelength. At any one time, single-wavelength emitters can make 4 connections and multi-wavelength emitterscan make 12 connections. The network is in principle scalable to an infinitenumber of connections. In practice, this number is severely limited by lithog-raphy and etching precision. For a 5nm tolerance on the size of the microdisk,corresponding to state of the art CMOS process technology, the maximum sizeof the network is 8 × 8.Table 2.3. Truth table for optical network on chip T1 T2 T3 T4 M1 λ2 λ3 λ1 λ4 M2 λ3 λ4 λ2 λ1 M3 λ1 λ2 λ4 λ3 M4 λ4 λ1 λ3 λ2 The basic element of the network is an optical filter, described in the nextsection. The ports 1 − 4 correspond to inputs/outputs of the optical filter.Its operation is the same as an electronic cross-bar: the cross function (outputin 4) is activated when the injected wavelength in 1 does not correspondto a resonant ring wavelength and the bar function is activated (output in 3)when the injected wavelength in 1 corresponds to a resonant ring wavelength.
  • 35 IPM1 M1 T1 IPT1 1 3 IPM2 M2 T2 IPT2 2 4 IPM3 M3 T3 IPT3 1 3 IPM4 M4 T4 IPT4 #1 #3 master master passive optical i i = n target target IP blocks interface network on chip interface IP blocks (driver, elementary optical n (detector, laser) filter operation receiver) #2 #4 i = n Figure 2.8. Architecture of 4x4 optical network on chipOperation is symmetrical: the same phenomena happens if the wavelengthinjection is placed in the port Microresonators Microring resonators are ideal device candidates for integrated photonic cir-cuits. Because they render possible the addition or extraction of signals froma waveguide based on wavelength in a WDM flow, they can be considered asbasic building blocks to build complex communication networks. The use ofstandard SOI technology leads to high compactness (structures with radii assmall as 4µm have been reported) and the possibility of low-cost photonic in-tegration. Figure 2.9 shows the structure of an elementary add-drop filter basedon microring resonators. The size of the structure is typically a few hundredµm2 . It consists of two identical disks evanescently side-coupled to two signalwaveguides which are crossed at near right angles to facilitate signal direc-tivity. The microdisks make up a selective structure: the electromagnetic fieldpropagates in the rings for discrete propagation modes corresponding to specificwavelengths. The resonant wavelengths depend on geometric and structural pa-rameters (indices of the substrate and of the microrings, thickness and diameterof the disks). The basic function of a microresonator can be thought of as a wave-length-controlled switching function. If the wavelength of an optical signal passingthrough a waveguide in proximity to the resonator (for example injected at port 1) is close enough to a resonant wavelength λ1 (tolerance is of the order of a fewnm, depending on the coupling strength between the disk and the waveguide),then the electromagnetic field is coupled into the microrings and then out alongthe second waveguide (in the example, the optical signal is transmitted to the
  • 36 #1 #3 = r #3 #4 10um #1 r i #2 i r = r #2 r #4 30um Figure 2.9. Micro-disk realisation of an add-drop filteroutput port 3, as shown in fig. 2.10a). If the wavelength of the optical signaldoes not correspond to the resonant wavelength, then the electromagnetic fieldcontinues to propagate along the waveguide and not through the structure (inthe example, the optical signal would then be transmitted to the output port 4,as shown in fig. 2.10b). This device thus operates as an elementary router, thebehaviour of which is summarised in the table in fig. 2.9.Figure 2.10a. FDTD simulation of add- Figure 2.10b. FDTD simulation of add-drop filter in on-state drop filter in off-state First structures have been realised and preliminary results are promising.Fig. 2.11a shows an IR photograph of the structure in the cross state (top) andin the bar state (bottom), while fig. 2.11b represents the transmission coefficienton the cross output: the transmitted power on the cross output reaches 100%for wavelengths corresponding to the resonant frequencies of the microdisk.2.5.2 Microsource lasers From the viewpoint of mode field confinement and mirror reflection, mi-crodisk lasers operate on the principle of total internal reflection, as opposed tomultiple reflection, as is the case in VCSELs for example. This fact gives thistype of source two distinct advantages over VCSELs for on-chip optical inter-connect. Firstly, light emission is in-plane (as opposed to vertical), meaning
  • 37Figure 2.11a. Infra-red photograph of Figure 2.11b. Transmission coefficient onstructure in both cross (top) and bar (bot- cross output for varying wavelengthtom) statesthat emitted light can be injected directly into a waveguide with minimum loss[6]. Secondly, for communication schemes requiring multiple wavelengths,it is easier from a technological point of view to control the radius of such adevice than it is to control the thickness of an air gap in a VCSEL. In any casesuch devices, to be compatible with dense photonic integration, must satisfy therequirements of small volume and high optical confinement, with low thresholdcurrent and emitting in the 1.3-1.6µm range. Although these devices are notas mature as VCSELs, they seem extremely promising for optical interconnectapplications. An overview of microcavity semiconductor lasers can be foundin [2].2.5.3 Demonstration of principle Behavioural models enable us to verify the operation of the 4 × 4 ONoC athigh level simulation. An injection of 4 wavelengths is realised (λ1 , λ2 , λ3 ,and λ4 ) at the port 1 at the same moment (shown in figure 2.12). The inputsignal format is a matrix. Figure 2.12 is a 3-dimensional representation withwavelength on the X-axis (representing the 4 channels), time on the Y-axis andpower (normalised) on the vertical axis. Each injected wavelength has twopulses (Gaussian) in time. The behavioural simulation analyses the 4 outputsT1 , T2 , T3 and T4 (T2 shown in fig. 2.12). As predicted in table 2.3, only λ3 isdetected at the output T2 . Figure 2.12. Simulation of 4x4 optical network on chip
  • 382.6 CONCLUSION Integrated optical interconnect is one potential technological solution to al-leviate some of the more pressing issues involved in moving volumes of databetween circuit blocks on integrated circuits. In this chapter, we have shownhow novel integrated photonic devices can be fabricated above standard CMOSICs, designed concurrently with EDA tools and used in clock distribution andNoC applications. The feasibility of on-chip optical interconnect is no longerreally in doubt. We have given some partial results to quantitatively demon-strate the advantages of optical clock distribution. Although lower power canbe achieved (of the order of a five-fold decrease), more work is required toexplore new solutions that benefit from advances both at the architectural andat the technological level. Also the existing basic building blocks need to beintegrated together to physically demonstrate on-chip optical links. Research iswell under way in several research groups around the world to do this. Lookingfurther ahead, the use of multiple wavelengths in on-chip communication net-works and in reconfigurable computing is an extremely promising and excitingfield of research.References[1] M. Amann, M. Ortsiefer, and R. Shau: 2002, ‘Surface-emitting Laser Diodes for Telecommunications’. In: Proc. Symp. Opto- and Microelec- tronic Devices and Circuits.[2] T. Baba: 1997, ‘Photonic Crystals and Microdisk Cavities Based on GaInAsP-InP System’. IEEE J. Selected Topics in Quantum Electronics 3.[3] Y. Cao, T. Sato, D. Sylvester, M. Orchansky, and C. Hu: 2000, ‘New Paradigm of Predictive MOSFET and Interconnect Modeling for Early Cir- cuit Design’. In: Proc. Custom Integrated Circuit Conference.[4] S. Cho et al.: 2002, ‘Integrated detectors for embedded optical interconnec- tions on electrical boards, modules and integrated circuits’. IEEE J. Sel. Topics in Quantum Electronics 8.[5] A. Filios et al.: 2003, ‘Transmission performance of a 1.5-µm 2.5-Gb/s directly modulated tunable VCSEL’. IEEE Phot. Tech. Lett. 15.[6] M. Fujita, A. Sakai, and T. Baba: 1999, ‘Ultrasmall and ultralow threshold GaInAsP-InP microdisk injection lasers: Design, fabrication, lasing charac- teristics and spontaneous emission factor’. IEEE J. Sel. Topics in Quantum Electronics 5.[7] M. Fujita, R. Ushigome, and T. Baba: 2000, ‘Continuous wave lasing in GaInAsP microdisk injection laser with threshold current of 40µA’. IEE Electron. Lett. 36.
  • 39[8] M. Ingels and M. S. J. Steyaert: 1999, ‘A 1-Gb/s, 0.7µm CMOS Optical Receiver with Full Rail-to-Rail Output Swing’. IEEE J. Solid-State Circuits 34(7).[9] I. Kimukin et al.: 2002, ‘InGaAs-Based High-Performance p-i-n Photodi- odes’. IEEE Phot. Tech. Lett. 26(3).[10] K. Lee et al.: 2001, ‘Fabrication of ultralow-loss Si/SiO2 waveguides by roughness reduction’. Optics Letters 26.[11] J. Liu et al.: 2002, ‘Ultralow-threshold sapphire substrate-bonded top- emitting 850-nm VCSEL array’. IEEE Phot. Lett. 14.[12] J. Morikuni et al.: 1994, ‘Improvements to the standard theory for pho- toreceiver noise’. IEEE J. Lightwave Technology 12.[13] I. O’Connor, F. Mieyeville, F. Tissafi-Drissi, G. Tosik, and F. Gaffiot: 2003, ‘Predictive design space exploration of maximum bandwidth CMOS photoreceiver preamplifiers’. In: Proc. IEEE International Conference on Electronics, Circuits and Systems.[14] A. Sakai, T. Fukazawa, and T. Baba: 2002, ‘Low Loss Ultra-Small Branches in a Silicon Photonic Wire Waveguide’. IEICE Tran. Electron. E85-C.[15] A. Sakai, G. Hara, and T. Baba: 2001, ‘Propagation Characteristics of Ultrahigh-∆ Optical Waveguide on Silicon-on-Insulator Substrate’. Jpn. J. Appl. Phys. – Part 2 40.[16] S. Schultz, E. Glytsis, and T. Gaylord: 2000, ‘Design, Fabrication, and Performance of Preferential-Order Volume Grating Waveguide Couplers’. Applied Optics-IP 39.[17] Semiconductor Industry Association: 2003, ‘International Technology Roadmap for Semiconductors’.[18] G. Tosik, F. Gaffiot, Z. Lisik, I. O’Connor, and F. Tissafi-Drissi: 2004, ‘Power dissipation in optical and metallic clock distribution networks in new VLSI technologies’. IEE Elec. Lett. 4(3).
  • 40Chapter 3NANOTECHNOLOGIES FOR LOW POWERJacques GautierCEA-DRT – LETI/D2NT – CEA/GREAbstract The conventional approach to improve the performance of circuits is to scale down the devices and technologies. This is also convenient to lower the power consumption per function. In this chapter, we overview the potential of nanotechnologies for this purpose, with emphasis on few-electron devices in the case of room-temperature operation. Other devices, especially carbon nanotube transistors, resonant tunnelling diodes and quantum cellular automata, are briefly discussed.Keywords: nanotechnologies; Single Electron Transistor; SET; molecular electronics; RTD; QCA; low power; Coulomb blockade3.1 INTRODUCTION In addition to packing-density increase and speed improvement, thedownscaling of technologies comes with a reduction of the powerconsumption per function. However this gain is offset by the tremendousincrease in the number of transistors per chip. A possible solution is to gofurther towards nano-scale devices where a lower amount of charge isneeded to code a bit. This is the basis of what is known as single electronics.The use of molecules could be a realistic way to fabricate these tiny devicesand other useful nanostructures. In this chapter we overview the potential of nanodevices for low powerelectronics with emphasis on few-electron electronics in the case of room-temperature (RT) operation. Other devices, especially carbon nanotubetransistors, resonant tunnelling diodes (RTD) and quantum cellular automata(QCA), are briefly discussed.
  • 413.2 SINGLE ELECTRONICS In CMOS circuits, the total power consumption is the sum of thedynamic power and of the contribution of leakages. For advancedtechnology generations the later is rapidly rising, but it is still less than theformer. So, we will focus on this dynamic power consumption which isgiven by the usual expression Pd = a ⋅ N gate ⋅ (C gate + Cint er )⋅VDD ⋅ f c 2 (1) where a is the activity factor, Ngate is the amount of gates, (Cgate + Cinter) isthe load capacitance, gate and interconnect contributions, and f is the clockfrequency. This equation shows that the power is proportional to the amountof charge in transistors and interconnects for coding a bit of information. Fordense circuits with local interconnects, the dominant contribution is usuallythe one related to the gate capacitance of transistors which can also beexpressed as Pd=a.Ngate.Q.VDD.fc, where Q is the channel charge. So there is astrong motivation to reduce it for power saving. This is currently obtained bythe downscaling of technologies. From the extrapolation of the historicaltrend and from the ITRS roadmap anticipation[1], we can expect a value ofonly 10-20 electrons for sub-10nm MOSFETs. This is much less than thehundreds to thousands of electrons present in current devices. Is it possibleto go still further, towards only one electron, using what is called a singleelectron transistor or SET [2]? That would be advantageous for powerconsumption, knowing that the reduction of power per function due to thescaling is more or less balanced by the tremendous increase of the number oftransistors per chip. However this gain would be effective only if thecapacitances of interconnects are not too large. Another factor in expression(1) is the electrostatic potential at which the charge Q is brought. At present,there is a strong incentive for reducing it. Whereas the supply voltage ofcurrent high performance circuits is in the range 1.2-1.8V, operation at only0.3V on experimental circuits has already been demonstrated [3], which isclose to the bottom limit anticipated by the ITRS. For a lower value thedevice is not in well defined On or Off states which results in either leakageor poor performance. What can be expected from SETs ? Before giving ananswer to this question, their properties and modes of operation are brieflyrecalled.3.2.1 Background on single electron transistors A SET is a device which comprises a Source and a Drain reservoir ofelectrons and a control gate, like MOSFETs. In between, there is an island
  • 42where carriers should be confined [2] (see Fig. 3-1). A common solution toobtain this effect is to insert tunnelling or potential barriers between thereservoirs and the island. This is the main structural difference fromMOSFETs, but it is essential for the operation of SETs. Due to thisconfinement, there is always an integer number of electron in the island.However, the probability to have a given amount of charge is a continuousfunction of the device bias, such that there is also a continuous variation ofthe average charge versus the external bias. RT>>RQ RT RT S D Cj Cj island G Cg Vg Figure 3-1. Schematics of a SET Provided that just one electron more or less has a significant effect on theelectrostatic energy of the device, it is shown that, for a given device bias,there are limited possible states of charge in the island [2]. Especially, thereare bias domains for which only one state of charge is possible. In this case,there is no exchange of charge with the electron reservoirs and the device isin Off state. This is the Coulomb blockade effect. For the other cases, thenumber of electron oscillates between the highest probable states of chargeleading to a flux of carriers between source and drain. For instance, when thetwo states n and n+1 are possible, the current is due to the repetition of thesequence: one electron coming from the source to the island then leaving theisland to the drain. As shown in Fig. 3-2, the electrical characteristics of SETs are verydifferent from those of MOSFETs. The ID(VG) curves have periodicoscillations of current and the output characteristics look like a resistance (orstaircase for non-symmetrical device) with a low drain voltage domainwhere the device is periodically Off and On as a function of VG. The periodof Coulomb Blockade Oscillations, CBO, is given by e/Cg. Between twosuccessive oscillations, the only difference is that the average number of
  • 43electron in the island is incremented or decremented by one. At a peak ofcurrent, two dominant states of charge have equal probability and, on theaverage, there is a half integer number of electron in the island. -7 -7 2 10 ID (A) 1,2 10 ) A ( -7 D 1 10 I -7 1,5 10 VD=0.4V -8 8 10 -7 -8 1 10 6 10 200mV 4 10 -8 Vg=0.45V -8 5 10 100mV -8 2 10 Vg=0.1V 20mV 0 0 0 0,5 1 1,5 2 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 Vg (V) Vd (V)Figure 3-2. Typical ID(VG) and ID(VD) characteristics of a SET. They have been obtained by simulation with the following parameters: Cj=0.1aF, Cg=0.2aF, RT=1MΩ, T=300K To observe the previous typical characteristics, there are two importantconditions to meet. Firstly, the charging energy, which is the electrostaticenergy increase due to the arrival of one electron in the island, should belarge in comparison to the thermal energy kT: e2 (2) EC = >> kT 2CΣ where e is the electron charge (absolute value), CΣ is the total capacitanceof the island, CΣ=2Cj+Cg, where Cj is the junction capacitance and Cg is thegate to island capacitance. For room-temperature operation, CΣ should beless than 0.3 aF (Ec=10 kT, T=300K), which requires an island smaller thana few nm. The second condition is related to the confinement of the electronwave function in the island, which is essential to quantize the charge in thisisland. The resistance of the tunnel barriers should exceed the quantumresistance RK=h/e2~25.8 kΩ. For the fabrication of SET’s, there are many different possibilities sinceany kind of conductive material can be used for the island, metallic as wellas semiconductor and even molecular. However silicon is advantageous forCMOS compatibility and also for the stability of devices [4].
  • 443.2.2 Designing a low VDD inverter With regard to the power consumption of digital circuits, we consider inthis part the case of a simple inverter, since this is a convenient reference tomake comparisons with CMOS. The design of a SET inverter has beendiscussed by many authors [5,6,7].They pointed out that, since there is onlyone kind of SET, the complementary action of the pull-up and pull-downdevices is not as easy to obtain as in CMOS where two types of transistorexist. A first solution is to choose the supply voltage in order that both ofthese devices are On or Off in a complementary way in the switching part ofthe transfer characteristic. An example of such situation is shown in Fig. 3-3.The shaded area displays the Coulomb blockage domains of the pull-up andpull-down transistors at zero temperature. Based on that, the transfercharacteristics has been schematically drawn. Contrary to CMOS, we canobserve that the voltage swing is less than rail-to-rail and that the DC currentis minimal at the transition point. Vout (V) 1 0,8 VDD 0,6 0,4 0,2 0 1,00 0,00 0,20 0,40 0,60 0,80 -0,2 -0,4 -0,6 Vin (V) Figure 3-3. Theoretical Coulomb blockade domains, also known as Coulomb diamonds, (shaded areas) at 0K, for the pull-down and pull-up SETs of an inverter. At RT they are a little narrower. Cj=0.1aF, Cg=0.2aF, VDD=0.53V. The bold line is a drawing of the transfer characteristics.
  • 45 Since a low VDD is advantageous for low power applications, we discussnow the possibility to minimize it for this simple SET inverter, takingaccount of the design constraints and aiming room-temperature operation:• Cg + 2.Cj < 0.3aF for RT operation (for Ec~10kT)• Cg / Cj > 1 for voltage gain• VDD = e / (Cg + Cj) for complementary action of transistors As a result, a very low VDD and RT operation would be difficult toachieve simultaneously. In fact, with the previous equations and for a ratioof gate to junction capacitances of 2, the minimum VDD would be equal to0.7V ! However, for temperatures above 0K, the switching of the SET fromOff to On state is not abrupt since there is an exponential variation of thecurrent, equivalent to the subthreshold current of MOSFET’s. Consequently,the real Coulomb blockade diamonds are narrower than those shown in Fig.3-3 and it is possible to reduce VDD. This is demonstrated in Fig. 3-4, wherethe DC voltage gain and the DC current at the transition point of an inverterhave been plotted versus VDD. Note also that the constraint on CΣ has been alittle relaxed. As thoroughly discussed by A. Korotkov [6], the acceptableVDD window is quite narrow. A too low VDD value would be detrimental forthe noise margin and for the speed since the DC current at the transitionpoint is exponentially decreasing with VDD. On the contrary, a higher valuewould increase the power consumption. -7 Voltage gain 1,6 10 Imin (A) 1,4 1,2 -8 10 1 V =e/(Cj+Cg) DD 0,8 -9 Cj=0.1aF 10 0,6 Cg=0.2aF R =1MΩ 0,4 T T=300K -10 0,2 10 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 V (V) DDFigure 3-4. DC voltage gain (solid line) and DC current (dashed line) at the transition point of a SET inverter at room temperature.
  • 46 To go further in reducing VDD, a solution is to add control gates to eachSET (Fig. 3-5). Based on this approach, NTT has demonstrated a quasi-CMOS operation inverter at a supply voltage as low as 20 mV8, which isvery advantageous for the power consumption, but in this case thetemperature was only 27K. The bias of the control gates shifts the CBO,making possible to select the optimal part of the ID(VG) characteristics ofeach SET for complementary action. In this way, the equivalent of two typesof transistors can be obtained, like for CMOS. In addition, their equivalentthreshold voltages can be tuned, balancing the influence of eventual parasitic(background) charges in the neighbourhood of the SET: C gc (3) ∆Vg = − ∆Vgc Cg To get a symmetrical transfer characteristic, from the Coulombdiamonds, it can be easily demonstrated that the sum of the control gatevoltages should be equal to VDD: Vgcss + Vgcd d = VDD (4) As a result there is one more degree of freedom to design an inverter, incomparison to the case without control gates. That gives flexibility to fix thevalue of VDD. In fact, there is now one optimal supply voltage, leading tocomplementary states of pull-up and pull-down transistors, for each bias ofthe control gates. Taking equation (4) into account, it is given by: e − 2CgcVgcss (5) VDDopt = Cg + C j There is a consequent reduction of VDDopt thanks to the control gates, butit is important to note that the constraint on the total capacitance (equation 2)should also take account of the contribution of Cgc : CΣ=2Cj+Cg+Cgc A drawback of this approach is the requirement of extra lines to distributethe control gate voltages. However, this can be avoided in the particular casewhere Vgcss=VDD and Vgcdd=0V (VSS=0V). For this condition, the optimumvalue of VDD is given by: e (6) VDDopt = C g + C j + 2C gc
  • 47 As discussed previously for the case without control gates, at RT theCoulomb blockade area of SET is narrower than at 0K which makes possiblea reduction of VDD or a change of bias of the control gates for a given VDD.However, it also results in a change of the SET current which may affect thespeed of circuits. Consequently, there is a design trade-off. To illustrate it, inFig. 3-5 we have plotted the variations of the DC voltage gain and of thepropagation delay along a chain of SET inverters versus the DC current atthe transition point of the transfer characteristics. The shaded area shows themost advantageous design window. In this example, the load capacitance isequal to 0.5fF, but for another value the design window would be the same,since the propagation delay directly scales with this capacitance. This is adifference with CMOS where the dominant load capacitance of dense logicis due to the gate capacitance of MOSFETs. Here, the gate capacitance ofSETs is extremely small and the dominant load capacitance comes from thelocal interconnects. In fact the later should be much larger than e/2VDD toavoid any detrimental effects of the shot noise. Regarding the dynamic power consumption of SET logic, as long as aCMOS output buffer is not implemented, the major contribution would alsocome from the load capacitance due to the interconnects. -8 2 5 10 propagation delay (s) DC voltage gain Master Equation Cj=0.05aF C =0.1aF G -8 G C =0.1aF 4 10 V GC C =0.5fF 1,5 load VDD -8 R =1MΩ 3 10 T CG VGdd τ -8 Vout p 2 10 Vin C 1 G CL V =0.3V -8 VGss DD 1 10 T=300K V +V =V GCss GCdd DD 0,5 0 -10 -9 -8 -7 10 10 10 10 DC current @ transition point (A) Figure 3-5. Variations of the DC voltage gain (solid line) and of the propagation delay along a chain of SET inverters (dashed line) versus the DC current at the transition point of the transfer characteristics. VDD=0.3V. The control gate voltages are varied as follow:0.7V<VGCss<0.1V and VGCss+VGCdd=VDD. The simulations are performed using a model of the SET based on the solution of the Master Equation [2]. The shaded area is the design window for sufficient voltage gain and speed.
  • 48 An example of switching characteristics is shown in Fig. 3-5 forVDD=0.3V, T=300K and a load capacitance of 0.5fF. The correspondingswitching energy is 5.2x10-2fJ. In the same figure, transfer characteristicshave been plotted for a lower supply voltage, showing that a voltage gainhigher than 1 can be obtained down to VDD=0.2V at RT. -9 0,5 5 10 Vin, Vout (V) 0,2 Output (V) I I Current (A) pull-down pull-up -9 0,4 4 10 0,15 -9 0,3 3 10 0,1 -9 0,2 2 10 T=300K -9 0,05 0,1 1 10 V V out in T=150K 0 0 0 -8 -7 -7 -7 0 5 10 1 10 1,5 10 2 10 0 0,05 0,1 0,15 0,2 time (s) input (V) Figure 3-5. On the left, switching characteristics for a SET inverter with control gates.VDD=0.3V, T=300K, CL=0.5fF, Cj=0.05aF, Cg= Cgc =0.1aF, RT=1MΩ, Vgcss=0.3V, Vgcdd=0. On the right, transfer characteristics for the same capacitances and VDD=0.2V.3.2.3 Designing gates with increased functionality Another approach to lower the power consumption is to build logic gateswith increased functionality in order to reduce the count of transistor neededto obtain a given function. This can be done by taking advantage of both theexistence of CBO and the possibility to design SETs with multiple inputs[9]. The principle is to choose the logic levels such as the multiple inputsSETs are biased at either the minima or at the peaks of current, dependingon the combination of input signals. This is illustrated in Fig. 3-7 in the caseof a double input X-OR function. The logic level "1" is equal to the CBOperiod of the equivalent single input SET, e/(2.Cin), where Cin is the inputgate capacitance. From this equivalent SET, it is obvious that the device is
  • 49Off when both of the inputs are either "0" or "1" and that the device is Onwhen one and only one of the inputs is "1". -8 3 10 ID (A) -8 V =100mV T=300K D 2,5 10 VDD Cin -8 2 10 VA -8 1,5 10 VB -8 1 10 Cin -9 5 10 VGeff 0 CGeff 0 0,5 1 1,5 2 Vgeff (V) A ="1" C Geff = 2Cin A="0" A ="1" and X-OR and VA + VB B="0" B ="1" B ="1" VGeff = 2Figure 3-7. Principle of design of a X-OR gate with a double input SET. The output current is a X-OR function of VA and VB. The figure shows also the equivalent input circuit and associated equations. This can be used to design pass-gate logic functions, as demonstrated byY. Ono [9]. For instance, for an input current signal C in the previous X-ORgate, the output pass current is given by C.(A ⊕ B). Furthermore, one of theinputs of this gate, B for instance, can be viewed as a control input, leadingto the output pass current C.A if B is "1". Applying this technique to a gatecomprising such a control input in addition to the inputs A and B, we get thefunction C.( A ⊕ B) for the output current, when the control input is "1" (seeFig. 3-8). This way, cascading SET structures, NTT has designed a 4-b adderwith only 40 SETs for operation at 30K9. In comparison with CMOS thereare less transistors and no crossing of pass signal routes thanks to the highfunctionality of the SET gates. Furthermore, there is a low level signal on thepass route. Consequently, a lower dynamic power is expected. Nevertheless,this power gain is not yet evaluated.
  • 50 C.(A⊕B) A Control gate B CFigure 3-8. Design of complex gates using multiple input SETs. These gates can be cascaded. Moreover, there are important issues, especially about the control of thephase of CBO. Since it will be probably impossible to avoid the existence ofany parasitic charges, charge-tolerant solutions are required. A first approachconsists in incorporating redundancy into the circuit design in order toreplace the defective gates by reconfiguration [9]. This is valuable only if areasonable amount of spares is needed and if the area-overhead is not toolarge. Another solution would be to balance the influence of parasitic offsetcharges by opposite charges stored near the island of the SET. This concepthas been demonstrated in the case of SET in which nanostructures have beenembedded [10-12]. The resulting device is a merge of a SET and of a NonVolatile Memory function. Further, a feedback loop can be implemented toautomatically control the phase of CBO [13]. The loop is closed to adjust theamount of charges in a memory node then it is opened for the use of thedevice. There are other potential applications about the possibility to tune andmemorize the phase of CBO. A first example has been the demonstration ofa hybrid SET-MOSFET gate which can be programmed to be inverting ornon-inverting [11]. This feature has been obtained thanks to a SET activedevice which can operate either in a positive or in a negativetransconductance region, depending on the amount of charge stored in anearby nanostructure. In this case, the SET was fabricated in a very thinundulated SOI film in which a narrow source-drain percolation channel andan electron pocket working as a memory node can be naturally formed for arange of bias. In the hybrid gate, the MOSFET was just used as a load. Sincethe output voltage swing was only 10mV, an output buffer has beenimplemented. The reproducibility of the structure is not obvious, but RToperation and peak-to-valley current ration (PVCR) as high as 102 wereobtained. The most important is the concept of programmable logic which isfeasible with SET based devices, since it has a high potential for low-powerand high packing density. The design of SET programmable logic array(PLA) has been also reported by K. Uchida [11].
  • 51 It is important to note that many other functions can be designed withfew-electron devices, taking advantage of their specific features. Especially,several memory structures that are promising for low power consumptionhave been reported [14-17]. For spiking neuron circuits, it has been proposedto combine NVM MOSFET devices and single-electron circuit based onmultinanodot floating-gate arrays [18]. Also, some analog applications anddevices have been studied, like CCD [8], ADC [19], metrology [20] andNEMS [21]. However, although some have been demonstrated, most of themare still at the proof of concept level.3.3 MOLECULAR ELECTRONICS For the fabrication of SETs, any kinds of conducting materials can beused. Whereas the basic research was done on metallic SETs [2], circuitdemonstrations are performed mainly on silicon [8-13], for complementarywith MOSFETs and to benefit of the huge investment in silicon technology.However, it could be advantageous to use molecules for real applicationsdue to the size requirement discussed previously and because thereproducibility of nanoscale structures is very challenging. In addition, theload capacitance of circuits should be very low, for power consumption andspeed considerations, which implies short and narrow interconnects. Themost promising way to achieve it is the bottom-up approach, using naturallyformed tiny structures or self-assembling methods. The best example is thecarbon nanotube (CNT) which can be used to fabricate FETs [22-23], SETs[24], interconnects [25] and even non volatile memory arrays [26]. CNTs are long cylinder of carbon atoms consisting of rolled-up sheets ofgraphite. For Single Wall CNTs the diameter is as small as 1-5 nm.Depending on their chirality, they are semiconductor or metallic materials.Their mobility is much higher than the one of silicon, and a ballistictransport has been demonstrated for lengths less than a few hundreds of nm,but the subthreshold characteristics of CNFETs are not better than those ofMOSFETs. Worldwide, several teams are conducting research on theselective growth or deposition of CNTs that would have the right chiralityand on the evaluation of CNFETs as potential candidate to replaceMOSFETs in the future. For low power applications, thanks to theirexcellent transport properties [22], it should be possible to reduce the gateoverdrive and VDD, while meeting the ITRS specifications [1]. Different kinds of molecules are also currently investigated to makenanometer-scale electronic components and circuits, but a single moleculetransistor has not yet been obtained. To date, one of the most advancedachievement is a 1µm2 64 cells crossbar matrix fabricated by HP Labs [27]in which the switching units are bundle of rotaxane molecules. The operation
  • 52of such molecules is not yet clear and others mechanisms like the formationof tiny filaments across the molecule gap between the electrodes couldexplain the switching [28]. However, on the long term, this research subjecthas a great potential for high density, low cost and probably ultra low powerelectronics. True single molecule device will require interconnects at a similar scale.This is also essential to reduce parasitic capacitances and the powerconsumption. Since the needed resolution is far beyond the possibility oflithographic tools, including NGL, the solution will come from the bottom-up approach. An example is the realization by Caltech of a Pt nanowirelattice with width and pitch of 8 nm and 16 nm respectively [29]. Biologycan also come to the rescue for the self-assembly of nano-circuits [30]. Avery different approach, also mitigating the arduous task of nanoscalepatterning, is the concept of self-assembled nanocells proposed by J. Tour[31]. These nanocells are disordered arrays of metallic islands that areinterlinked with molecules and that are accessed by metallic input/outputleads. Switching-type functions have been observed, but like for the work ofHP [27], the creation and dissolution of metal filaments is probablyresponsible for the behaviour. In fact, the behaviour of electrically activemolecules is strongly influenced by surrounding electrodes and othermaterials, which make a difference between molecular nanotechnology andbulk or solution-phase chemistry.3.4 DISCUSSION For CNT devices, as well as for nano MOSFETs, the supply voltagereduction is dependent upon the effects of subthreshold leakage on the staticpower, leading to trade-off with the speed of circuits. For SETs, thesteepness of the On-Off switching is not better, but they offer an increasedfunctionality and low charge operation. However, there are important issues,especially about the sensibility to offset charges and the fabrication of nano-scale structures with sufficient level of reproducibility, which require a lot ofwork. Although it is not yet clear if they could achieve a lower VDD, there areother candidates, like the resonant tunnelling diodes (RTD). Their operationis based on electron transport via discrete energy levels in double barrierquantum well structures, leading to the existence of a negative differentialresistance. This implies the fabrication in suitable materials and a perfectcontrol of the geometry, since the output characteristics are extremelysensitive to the dimensions. A promising approach is the implementation ofRTD along semiconductor nanowires [31]. There are also prospective
  • 53studies for a molecular version and for structures mixing Coulomb blockadeand resonant effects [32]. One of the most important features of nanodevices, especially formolecular ones, is their size. That offers the possibility to lower the powerconsumption by parallel processing. For instance, consider two blocks oflow capacitance molecular devices doing the same task as one block ofconventional devices, but at half the clock frequency. The Ngate . fc productbeing unchanged, equation 1 shows that the power consumption is directlyrelated to the C.V [2] product, the gate switching energy, which can bestrongly reduced thanks to lower capacitances and to the possibility to havedevices with lower On current, since fc is divided by 2 in this case. Going further, quantum-dot cellular automata (QCA) is an attractiveapproach, yet speculative, to reduce the power consumption, since there isno flow of current but only Coulomb interactions [33]. The principle is toencode binary information by charge configuration in electrostaticallycoupled cells in which there are two extra electrons. It has been shown that aclock field is needed to control the direction of propagation of informationalong the cells and to enable power gain. This clock could also be used for aquasi-adiabatic switching, leading to extremely low power consumption. Todate, experimental demonstrations are performed at low temperature onmetallic structures, but molecular implementations are being investigated inview of RT operation [34]. References[1] Semiconductor Industry Association, International Technology Roadmap for Semiconductors 2001 Edition, http://public.itrs.net, 2003[2] H. Grabert and M. H. Devoret, Single charge tunnelling Coulomb blockade phenomena in nanostructures, volume 294 of NATO ASI Series B, Plenum Press New York and London, 1992[3] T. Douseki, T. Shimamura, N. Shibata, A 0.3V 3.6GHz 0.3mW frequency divider with differential ED-CMOS/SOI circuit technology, in Proc. ISSCC, February 2003[4] N. M. Zimmerman and W. H. Huber, Excellent charge offset stability in a Si-based single-electron tunneling transistor, APL Vol. 79, N.19, pp. 3188-3190, 2001[5] J. R. Tucker, Complementary digital logic based on the Coulomb blockade, JAP 72 (9), 1, pp. 4399-4413, 1992[6] A. N. Korotkov, R. H. Chen and K. K. Likharev, Possible performance of capacitively coupled single-electron transistors in digital circuits, JAP 78 (4), pp. 2520-2529, 1995[7] M-Y. Jeong, B-H. Lee and Y-H. Jeong, Design considerations for low-power single- electron transistor logic circuits, JJAP. Vol.40, pp. 2054-2057, 2001[8] Y. Takahashi, Y. Ono, A. Fujiwara and H. Inokawa, Silicon Single-Electron Devices for logic applications, in Proc. ESSDERC September 2002, Florence, pp. 61-68[9] Y. Ono, H. Inokawa and T. Takahashi, Binary adders of multigate Single-Electron Transistors: specific design using Pass-Transistor Logic, IEEE Trans. on Nanotech. Vol.1 pp. 93-99, 2002
  • 54[10] N. Takahashi, H. Hishikuro and T. Hiramoto, A directional current switch using silicon Single Electron Transistors controlled by charge injection into silicon nano-crystal floating dots, in Proc. IEDM, pp.371-374, 1999[11] K. Uchida, J. Koga, R. Ohba and A. Toriumi, Programmable Single-Electron Transistor logic for future low-power intelligent LSI: proposal and room-temperature operation, IEEE Trans. on Elec. Dev. Vol.50, pp.1623-1630, 2003[12] G. Molas, X. Jehl, M. Sanquer, B. de Salvo, M. Gely, D. Lafond and S. Deleonibus, Manipulation of periodic Coulomb Blockade Oscillations in ultra-scaled memories by single electron charging of silicon nanocrystals floating gates, Silicon Nano Workshop, Honolulu, June 2004[13] K. Nishiguchi, H. Inokawa, Y. Ono, A. Fujiwara and Y. Takahashi, Automatic control of the oscillation phase of a Single-Electron Transistor, IEEE EDL25 (1), pp. 31-33, 2004[14] K. Yano, T. Ishii, T. Hashimoto, T. Kobayashi, F. Murai and K. Seki, Room- Temperature Single-Electron Memory, IEEE Trans. on Elec. Dev. Vol.41, NO.9,pp.1628-1638, 1994[15] Z. A. K. Durrani, A. Irvine and H. Ahmed, Coulomb blockade memory using integrated Single-Electron Transistor/Metal-Oxide-Semiconductor transistor gain cells, IEEE Trans. on Elec. Dev. Vol.47, pp.2334-2339, 2000[16] H. Sunamura, H. Kawaura, T. Sakamoto and T. Baba, Multiple-valued memory operation using a Single-Electron Device: a proposal and an experimental demonstration of a ten-valued operation, JJAP Vol. 41, pp. L93-L95, 2002[17] G. Molas, B. de Salvo, D. Mariolle, G. Ghibaudo, A. Toffoli, N. Buffet and S. Deleonibus, Single electron charging phenomena at room temperature in a silicon nanocrystal memory, in Proc. WODIM 2002, Grenoble[18] T. Morie, T. Matsuura, M. Nagata and A. Iwata, A multinanodot floating-gate MOSFET circuit for spiking neuron models, IEEE Trans. On Nanotechnology, Vol. 2, NO. 3, pp. 158-164, 2003[19] H. Inokawa, A. Fujiwara and Y. Takahashi, A multiple-valued logic and memory with combined Single-Electron and Metal-Oxide-Semiconductor transistors, IEEE Trans. on Elec. Dev. Vol.50, NO.2, pp. 462-470, 2003[20] H. E. van den Brom et al., Counting electrons one by one – overview of a joint european research project, IEEE Trans. on Inst. and Meas. Vol. 52, NO.2, pp. 584-588, 2003[21] S. Mahapatra, V. Pott, S. Ecoffey, A. Schmid, C. Wasshuber, J. W. Tringe, Y. Leblebici, M. Declercq, K. Banerjee and A. Ionescu, SETMOS: a novel true hybrid SET-CMOS high current Coulomb Blockade Oscillation cell for future nano-scale analog ICs, in Proc. IEDM 2003, pp. 703-706[22] A. Javey, H. Kim, M. Brink, Q. Wang, A. Ural, J. Guo, P. McIntyre, P. McEuen, M. Lundstrom and H. Dai, High-K dielectrics for advanced carbon-nanotube transistors and logic gates, Nature Materials, Vol 1, pp. 241-246, December 2002[23] P. Avouris, Carbon nanotube electronics, Chemical Physics, 281 (2002), pp. 429-445[24] K. Matsumoto, S. Kinoshita, Y. Gotoh, K. Kurachi, T. Kamimura, M. Maeda, K. Sakamoto, M. Kuwahara, N. Atoda and Y. Awano, Single-Electron Transistor with ultra-high Coulomb energy of 5000K using position controlled grown carbon nanotube as channel, JJAP Vol.42 Part 1 N°4B, pp. 2415-2418, 2003[25] J. Li, Q. Ye, A. Cassell, H. T. Ng, R. Stevens, J. Han and M. Meyyappan, Bottom-up approach for carbon nanotube interconnects, APL Vol. 82, N°15, pp. 2491-2493, 2003[26] Rueckes, K. Kim, E. Joselevich, G. Y. Tseng, C-L. Cheung and C. Lieber, Carbon nanotubes-based nonvolatile Random Access Memory for molecular computing, Science, Vol. 289, pp. 94-97, 7 July 2000
  • 55[27] Y. Chen, G-Y. Jung, D. A. Ohlberg, X. Li, D. R. Stewart, J. O. Jeppesen, K. A. Nielsen, J. F. Stoddart and R. S. Williams, Nanoscale molecular-switch crossbar circuits, Nanotechnology 14 (2003) 462-468[28] R. F. Service, Next-generation technology hits an early midlife crisis, Science Vol. 302, pp. 556-559, 24 October 2003[29] N. Melosh, A. Boukai, F. Diana, B. Gerardot, A. Badolato, P. M. Petroff and J. R. Heath, Ultrahigh-density nanowire lattices and circuits, Science, Vol. 300, pp.112-115, 4 April 2003[30] P. Fairley, Germs that build circuits, IEEE Spectrum, pp. 37-41, November 2003[31] M. T. Björk, B. J. Ohlsson, C. Thelander, A. I. Persson, K. Deppert, L. R. Wallenberg and L. Samuelson, Nanowire resonant tunneling diodes, APL, Vol. 81, N°23, pp. 4458- 4460, December 2002[32] M. Saitoh and T. Hiramoto, Room-temperature operation of highly functional Single- Electron Transistor logic based on quantum mechanical effect in ultra-small silicon dot, in Proc. IEDM IEDM 2003, pp. 753-756[33] G. Bernstein, Quantum-dot Cellular Automata: computing by field polarization, in Proc. DAC 2003, June 2-6, Anaheim (CA), pp. 268-273[34] C. Lent and B. Isaksen, Clocked molecular Quantum-dot Cellular Automata, IEEE Trans. on Elec. Dev. Vol.50, NO.9, pp. 1890-1896, 2003
  • 56Chapter 4STATIC LEAKAGE REDUCTION THROUGHSIMULTENEOUS VT/TOX AND STATE ASSIGN-MENTDongwoo Lee, Bo Zhai, David Blaauw and Dennis SylvesterUniversity of Michigan, Ann ArborAbstract: Standby leakage current minimization is a pressing concern for mobile applica- tions that rely on standby modes to extend battery life. In this paper, we propose new leakage current reduction methods in standby mode. First, we pro- pose a combined approach of sleep-state assignment and threshold voltage (Vt) assignment in a dual-Vt process for subthreshold leakage (Isub) reduction. Sec- ond, for the minimization of gate oxide leakage current (Igate) which has become comparable to Isub in 90nm technologies, we extend the above method to a combined sleep-state, Vt and gate oxide thickness (Tox) assignments approach in a dual-Vt and dual-Tox process to minimize both Isub and Igate. By combining Vt or Vt / Tox assignment with sleep-state assignment, leakage cur- rent can be dramatically reduced since the circuit is in a known state in standby mode and only certain transistors are responsible for leakage current and need to be considered for high-Vt or thick-Tox assignment. A significant improve- ment in the leakage/performance trade-off is therefore achievable using such combined methods. We formulate the optimization problem for simultaneous state/Vt and state/Vt/Tox assignments under delay constraints and propose both an exact method for its optimal solution as well as two practical heuristics with reasonable run time. We implemented and tested the proposed methods on a set of synthesized benchmark circuits and show substantial leakage current reduc- tion compared to the previous approaches using only state assignment or Vt assignment alone.Keywords: Leakage current, reduction, performance, dual threshold voltage, oxide thick- ness, algorithm.4.1 INTRODUCTION There is a growing need for high-performance and low-power system,especially for portable and battery-powered applications. Since these applica-tions often remain in stand-by mode significantly longer than in active mode,
  • 57their stand-by (or leakage) current has a dominant impact on battery life.Standby mode leakage current reduction therefore has been a concern forsome time and a number of such methods have been proposed to address thisproblem [1]-[7][9]-[18]. However, with continued process scaling, lower sup-ply voltages necessitate reduction of threshold voltages to meet performancegoals and result in a dramatic increase in subthreshold leakage current. Newmethods for reducing the leakage current in standby mode are therefore criti-cally needed. In dual-Vt technology, the MTCMOS approach [1] was proposed where ahigh-Vt sleep transistor is inserted between the power supply and the circuitlogic. In standby mode, this sleep transistor is turned off which dramaticallyreduces leakage due to its high-Vt. However, the method requires routing ofan additional set of power supply lines in the layout as well as substantiallysized sleep transistors to maintain good supply integrity and circuit perfor-mance [2]. Also, special latches that maintain state in standby mode need tobe used [3]. In addition, the method does not scale well into sub-1V technolo-gies due to the increased delay penalty for the high-Vt sleep device [4]. A different approach to standby mode leakage reduction has been pro-posed that leverages the state dependence of a leakage current due to the so-called stack effect [5][6]. In [7], the circuit input state that minimizes leakagecurrent is determined and special flip-flops are inserted in the design to pro-duce this state in standby mode. The flip-flops in the design are modified toproduce a predetermined state in standby mode while also maintaining thepreviously latched state. The required modification to a flip-flop is minor andcan be incorporated in the feedback path of the slave latch with minimalimpact on performance [8]. In general, determining the minimum sleep stateis a difficult problem due to the inherent logic correlations in the circuit. How-ever, a number of efficient heuristics for this problem have been proposed[9][10]. The limitation of this approach is that for larger circuits, the reductionin leakage current is typically only in the range of only 10 to 30% [9]. The above techniques are aimed primarily at subthreshold leakage currentreduction which has been the dominant component of leakage in CMOS tech-nologies to date. However, in 90nm technologies the magnitude of gatetunneling leakage, Igate, in a device is comparable to the subthreshold leakage,Isub, at room temperature. With difficulties in achieving manufacturable high-k insulator solutions to address the gate leakage problem, the burden addressthis problem is primarily on circuit designers and EDA tools. As a result,there has been recent work in the area of gate leakage analysis and reductiontechniques including pin reordering, PMOS sleep transistors, and the use ofNAND implementations rather than NOR implementations [11]-[13]. Also,the MTCMOS technique was extended to combat gate leakage by using a
  • 58thick-oxide I/O device with a larger gate drive than the logic transistors as theinserted sleep transistor [14]. Another previous approach to leakage reduction that targets only sub-threshold leakage is to use individual assignment of transistor thresholdvoltages in a dual-Vt process [15]-[18]. In these approaches, the trade-offbetween high-Vt transistors with low leakage/low performance and low-Vttransistors with high leakage/high performance is exploited. Circuit paths thatare non-critical are assigned high-Vt while critical circuit portions are givenlow-Vt assignments. The method therefore provides a trade-off between cir-cuit performance and leakage reduction. It was demonstrated that with amodest performance reduction of 5–10%, significant reduction of 3-4X inleakage could be obtained over a circuit with all low-Vt transistors [17]. Inthese approaches, high/low-Vt assignments are performed without knowledgeof the states of the circuit. Therefore, in order to obtain sufficient leakagereduction under all possible circuit states, all or most of the transistors in aparticular gate must be set to high-Vt and hence the gate incurs a substantialperformance degradation. While such dual-Vt processes have been commonplace for several genera-tions, the availability of multiple oxide thicknesses in a single process hasonly become relevant at the 90nm node due to the rise of Igate [19]. Given aprocess technology with dual oxide thicknesses for logic devices, the dual-Vtapproach can be easily extended to also consider gate leakage by assigningthick-oxide transistors to non-critical paths as well. However, similar to thedual-Vt assignment approach, a simultaneous dual-Vt and dual oxide thicknessassignment with unknown states of the circuit will set all or most of the tran-sistors in a particular gate to both high-Vt and thick-oxide, to ensure that underall possible circuit states in standby mode leakage current is acceptable. How-ever, transistors that are simultaneously assigned a high-Vt and a thick-oxidehave a dramatic delay penalty compared to low-Vt transistors with thin oxide.Therefore, this approach carries with it a significant delay penalty for processtechnologies where both Isub and Igate need to be addressed. In this paper, we therefore propose new methods to reduce standby modeleakage current. We can divide our new methods into two categories: 1)simultaneous dual-Vt and sleep state assignment for Isub reduction for technol-ogies in which Isub is dominant in standby mode and 2) simultaneous dual-Vt,dual oxide thickness and sleep state assignment for both Isub and Igate minimi-zation for technologies which have comparable amount of Igate to Isub. First,we combine the concepts of Vt assignment and sleep state assignment. Thisapproach is based on the key observation that, given a known input state for agate, the leakage of that gate can be dramatically reduced by setting only asingle OFF-transistor on each path from Vdd to Gnd to high-Vt. Since all other
  • 59transistors in the gate are kept at low-Vt and continue to have high drive cur-rent, the performance degradation is limited while significantly gain inleakage current is obtained. This approach therefore provides a much bettertrade-off between leakage and performance compared to Vt assignment withunknown input state where most or all of the transistors must be set to high-Vtbefore a significant improvement in the leakage current is observed. The linkbetween the effectiveness of Vt assignment and state assignment was previ-ously observed for Domino logic [8], since these circuits are by their ownnature in a known state in standby mode. However, we extend this concept togeneral CMOS circuits by actively controlling the circuit state in standbymode, thereby dramatically increasing the effectiveness of leakage reduction. The second proposed approach minimizes the total leakage current (Isuband Igate) by simultaneous assignment of sleep state, high-Vt and thick-oxidetransistors. In this approach, a key observation is that given a known inputstate, a transistor need not be assigned both a high-Vt and a thick oxide sinceIsub only occurs in transistors that are OFF while significant Igate occurs onlyin transistors that are ON. Furthermore, depending on the input state of a cir-cuit, only a subset of transistors need to be considered for either high-Vt orthick-oxide. Therefore, the impact on the delay of the gate is significantlyreduced while obtaining leakage reductions comparable to when all transis-tors are assigned to both high-Vt and thick-oxides. The proposed method iscompatible with existing library-based design flows, and we explore differenttrade-offs between the number of Vt and Tox variations for each library celland the obtained leakage reduction. In addition, we compare the obtainedleakage reduction when Vt (the first method) and Vt / Tox (the second method)assignments can be made individually for transistors in a stack as opposed towhen an entire stack is restricted to a uniform assignment due to manufactur-ing or area considerations. Since the circuit state / Vt and the circuit state / Vt / Tox assignments inter-act, it is necessary to consider their optimization simultaneously. The state / Vtand state / Vt / Tox assignment task is to find a simultaneous assignment thatminimizes the leakage current in standby mode while meeting a user specifieddelay constraint. We formulate this problem as an integer optimization prob-lem under delay constraints. The search space consists of all input state / Vtand input states / Vt / Tox assignments and hence is very large. Therefore, inaddition to an exact solution, we also propose a number of heuristics. The pro-posed methods are implemented on benchmark circuits synthesized using anindustrial cell library in 0.18 m technology for Isub minimization and in a pre-dictive 65nm technology for both Isub and Igate minimization. On average, theproposed Isub minimization method by simultaneous state / Vt assignmentapproach improves leakage current by a factor of 6X over the traditional
  • 60approach using Vt assignment only. The second proposed method that mini-mizes both Isub and Igate by simultaneous state / Vt / Tox assignment has anaverage leakage reduction of 5-6X over an all low-Vt and thin-oxide designsolution with a 5% delay range point and achieves more than a 2X improve-ment over the first proposed approach using Vt and state assignment only (i.e.,without dual-Tox). The remainder of this paper is organized as follows. In Section 4.2, we dis-cus the used leakage model and the characteristics of Isub and Igate leakagecurrent. In Section 4.3, we present the approach using simultaneous Vt andstate assignment for Isub leakage reduction. In Section 4.4, we present the sec-ond approach that also addresses Igate by performing simultaneous Vt, Tox,and state assignment. In Section 4.5, we present our results on benchmark cir-cuits and in Section 4.6 we present our conclusions.4.2 LEAKAGE MODEL AND CHARACTERISTICS In this section, we discuss our leakage current model and briefly reviewthe general characteristics of gate leakage current in CMOS gates. Since the proposed leakage optimization approach is library-based, we useprecharacterized leakage current tables for each library cell, with specificleakage table entries for each possible input state of a library cell. The pre-characterized tables were constructed using SPICE simulation with BSIM3models from 0.18 m technology for Isub minimization approach. In order torepresent both Isub and Igate components for the state / Vt / Tox assignmentapproach, BSIM4 models were used to generate the precharacterization oftables. The device simulation parameters were obtained using leakage esti-mates from a predicted 65nm processes [20], and had a gate leakagecomponent that was approximately 36% of the total leakage at room tempera-ture (at which all analysis is performed).1 (Detailed numbers will be shown atSection 4.5.2.) Different high- and low-Vt versions of a cell as well as Tox andVt versions of a cell will be explained further in Section 4.4.2. Also, the delayand output slope as a function of cell input slope and output loading werestored in precharacterized tables. The total gate leakage for a library cell consists of several different com-ponents, depending on the input state of the gate, as illustrated for the invertercell in Figure 4.1. The maximum gate tunneling current occurs when the inputis at Vdd and Vs = Vd = 0V for the NMOS device. In this case, Vgs = Vgd = Vddand the Igate is at its maximum for the NMOS device. At the same time, the1. Since this work aims at standby mode leakage, we expect junction temperatures during these idle periods to be lower than under normal operating conditions, making room temperature analysis more valid.
  • 61 Igate Isub Vdd Igate 0V Gnd Igate Vdd Igate Isub Figure 4.1. Inverter circuit with NMOS oxide leakage current.PMOS device exhibits substantial subthreshold leakage current. When theinput is at Gnd, the output rises to Vdd and Vgs = 0 while Vgd will become -Vddfor the NMOS device, resulting in a reverse gate tunneling current from thedrain to the gate node. In this case, tunneling is restricted to the gate-to-drainoverlap region, due to the absence of a channel. Since this overlap region ismuch smaller than the channel region, reverse tunneling current is signifi-cantly reduced compared to the forward tunneling current [21]. Note thatBSIM4 intrinsically considers this reverse tunneling current so it is includedin the precharacterized tables described above. When the input voltage is Gnd, the PMOS device also exhibits gate cur-rent from the channel to the gate since its Vgs = Vgd = -Vdd. The relativemagnitude of the PMOS gate current in comparison to the NMOS gate currentdiffers for different process technologies. If standard SiO2 is used as the gateoxide material, then the Igate for a PMOS device is typically one order of mag-nitude smaller than that for an NMOS device with identical Tox and Vdd[19][22]. This is due to the much higher energy required for hole tunneling inSiO2 compared to electron tunneling. However, in alternate dielectric materi-als, the energy required for electron and hole tunneling can be completelydifferent. In the case of nitrided gate oxides, in use today in a few processes,PMOS Igate can actually exceed NMOS Igate for higher nitrogen concentra-tions [23][24]. In this paper, we assume that standard SiO2 gate oxide materialis used and the PMOS gate current is negligible. However, the presentedmethods can be easily extended to include appreciable PMOS gate leakage aswell.4.3 SUBTHRESHOLD LEAKAGE REDUCTION4.3.1 Simultaneous Vt and State Assignment Consider the leakage and performance of the simple NAND2 circuitshown in Figure 4.2 under different input states and Vt assignments. It is clearthat given a particular input state, only those transistors that are OFF need to
  • 62 tp1 tp2 B Group 1 tn1 Group 2 A tn2 Group 3 Figure 4.2. The concept of groups for a NAND2 gatebe considered for high-Vt assignment as the ON-transistors are not leaking.For instance, in state AB = 01, only transistor tn1 needs to be considered forhigh-Vt assignment. Assigning other transistors to high-Vt will only decreasethe performance of the gate with no reduction in leakage current. On the otherhand, in state 11 both tp1 and tp2 must be assigned high-Vt in order to reduceleakage, since they are parallel devices. We can partition the transistors into so-called Vt-groups, corresponding tothe minimum sets of transistors that need to be set to high-Vt to reduce leak-age in a particular state assignment. For the 2-input NAND gate in Figure 4.2,three Vt-groups exist as shown. The concept of Vt-groups can be easilyapplied to more complex structures in which case it may be possible that atransistor belongs more than one Vt-group. It is clear that we can restrict our-selves to setting only entire Vt-groups to either high or low-Vt. By consideringonly Vt-groups, instead of individual transistors, we therefore significantlyreduce the number of possible Vt assignment and the optimization complexity.In Table 4.1, we show the leakage current for the NAND2 in Figure 4.2 fordifferent input states and Vt-group assignments. Column 3 shows the leakagecurrent when we use high-Vt for one or more Vt-groups that are OFF in a par-ticular input state. In column 4 and 5, the leakage current with all transistorsassigned to, respectively, high-Vt and low-Vt is shown. We can see that instates 01, 10, and 11 only a single Vt-group is a candidate for high-Vt assign- Table 4.1. Leakage current of NAND2 gate Input Assigned Leakage current [pA] State Group with Group Assign. with All High Vt with All Low Vt 2 24.9 00 3 9.8 7.2 286.7 2 and 3 7.2 01 2 26.6 26.6 1054.0 10 3 25.7 24.4 922.6 11 1 14.2 14.2 357.2
  • 63ment. Also, setting only this one Vt-group to high-Vt results in equal or nearlyequal leakage compared with the leakage when all transistors are assignedhigh-Vt demonstrating the effectiveness of the approach. In state 00, threehigh-Vt assignments are possible: group 2, group 3, and both group 2 and 3.However, the leakage current with both groups assigned to high-Vt is onlyslightly better than that with only one group set to high-Vt, and assigninggroup 3 to high-Vt reduces leakage somewhat more than assigning group 2 tohigh-Vt. Hence, it is clear that we need to only consider assignment of group 3to high-Vt without significant loss in optimality. Table 4.1 shows that the leakage current varies considerably as differentgroups associated with different input states are set to high-Vt. At the sametime, the impact of different high-Vt group assignments on the performance ofthe circuit must be considered. By setting only a single group to high-Vt, theperformance degradation is restricted to only a single signal transition direc-tion and is also reduced compared to high-Vt assignments where most or alltransistors are set to high-Vt. Therefore, the performance/power trade-off of Vtassignment with known input state is much improved compared with that withunknown input state. The input state of a gate effects which transition direction is degraded by ahigh-Vt group assignment to a gate. Also, the position of the high-Vt group ina stack of transistors changes the impact of a high-Vt group assignment on thedifferent input to output gate delays. Therefore, the input state of a gate mustbe chosen such that its associated high-Vt group results in the least degrada-tion of the critical paths in the circuit. However, only the input state of thecircuit as a whole can be controlled and the logic correlations of the circuitrestrict the possible assignments of gate input states. Therefore, selection ofthe circuit input state and of which gate is assigned a high-Vt group must bemade simultaneously to obtain the maximum improvement in leakage currentwith minimum loss in performance.4.3.2 Exact Solution to Vt and State Assignment The size of the input state space is 2n, where n is the number of circuitinputs. For each input state assignment, there are two possible Vt assignmentsfor each gate (one high-Vt group which is pre-determined by its input state,and all low-Vt). The total number of possible Vt assignment is therefore 2m,where m is the number of gates in the circuit and the total size of the searchspace is 2n+m. In order to find an exact solution to the problem, we developed an efficientbranch-and-bound method that simultaneous explores the state and Vt assign-ments and that exploits the characteristics of the problem to obtain efficientpruning of the search space to improve the run time. Due to the exponential
  • 64nature of the problem, an exact solution is only possible for very small cir-cuits. However, the exact approach is still useful as the proposed heuristicsare based on it. We use two types of branch and bound trees. The first branch-and-boundtree determines the input state of the circuit and is referred to as the state tree.The nodes of the state tree correspond to the input variables of the circuitinputs. Each node of the state tree is associated with a so-called gate treewhich is searched to determine the group Vt assignment. In other words, for astate tree with k nodes, there exist k copies of the gate tree. Each node in a par-ticular gate tree corresponds to a gate in the circuit, as shown in Figure 4.3.Each node has two fanout edges, representing the assignment of that gate withall low-Vt groups (left branch) or with one high-Vt group, as determined bythe input state of the gate (right branch). At the root of the state tree, the state of all input variables is unknown. Asthe algorithm proceeds down the tree, the state of one input variable becomesdefined with each level that is traversed. At each node in the state tree, a solu-tion of leakage current can be obtained by traversing the gate tree. Note thatthe gate tree may be traversed both with a completely known input state at thebottom of the state tree as well as with a partially or completely unknowninput state, at higher levels of the state tree. For each node in the state and gate tree, an upper and low bound on theleakage current is computed incrementally as explained in Section that early in the state tree the bounds on leakage will be very loose sincethe state of the circuit is only partly defined. As the algorithm traverses downthe state tree, the input state becomes more defined and the leakage boundsbecome closer. Similarly, the leakage bounds are very wide at the top of eachgate tree, as the Vt assignment of all gates are unknown, and becomes progres-sively tighter as the algorithm traverses down the tree. Only at the bottom ofboth the state tree and its associated gate tree do the upper and low bounds on Gate tree g1 L H g2 g2 L H L H State tree gm gm s1 L H L H 0 1 Sol. s2 s2 0 1 0 1 sn sn 0 1 0 1 Sol. Figure 4.3. State tree with gate tree at each node
  • 65leakage coincide. The algorithm first traverses down to the bottom of the treeand then returns back up, to traverse down unvisited branches in DFS manner.During the search, a tree branch is pruned when if it has a lower bound onleakage that is worse than the best upper bound on leakage that has beenobserved so-far. In addition to pruning based on leakage bounds, we alsocompute a lower bound on the circuit delay at each node in the gate tree tra-versal and prune all branches whose lower bound exceeds the specified delayconstraint. Computation of the delay bounds is also performed incrementallyand is discussed in Section Also, early in the state tree, computation of the exact minimum Vt assign-ment by traversing the gate tree is not meaningful since even at that bottom ofthe gate tree there is considerable uncertainty in the leakage current due to theunknown input state. Therefore, the gate tree is searched only partially at thehigher levels of the state-tree which results in slightly more conservativebounds, but an overall improvement in the run time of the algorithm. The gate tree is also searched in DFS manner and edges are pruned basedon the computed leakage bounds. During the downward traversal of the gatetree, the high-Vt branch is always selected, provide it meets the delay con-straint. This is due to the fact that the high-Vt branch always has less leakagecurrent than the low-Vt branch. Only if the lower bound on the delay of thehigh-Vt branch exceeds the delay constraint, is the low-Vt branch selected andis the high-Vt branch pruned. Finally, the gates in the circuit are assigned to nodes in the gate tree intopological order to enable incremental delay computation. Gates of equaltopological level are further sorted by decreasing leakage to improve thepruning of the search space. The input signals of the circuit are also assignedto nodes in the state tree in specific order. We want to place inputs whose stateassignment strongly influences the total leakage of the circuit near the top ofthe state tree. We estimate the influence of each input signal on the circuitleakage by taking the sum of the leakage current of all gates connected to theinput signal. This input variable ordering is similar to that used in [25]. Incremental leakage bound computation During the traversal of the gate tree, some of the gates will have a knownVt assignment and others, which have not been visited, will have an unknownVt assignment. As shown in Figure 4.4, a lower bound on the leakage is com-puted by assuming all unknown gates have a high-Vt group assignment and anupper bound is computed by assuming all unknown gates have a low-Vt groupassignment. As the high branch is taken in the downward traversal, only theupper bound is update (decreased) while when a low branch is taken, only thelower bound must be updated and is increased.
  • 66 UBi-1 = leak(g1~gi-1=known gates) + leak(gi~gn=L) LBi-1 = leak(g1~gi-1=known gates) + leak(gi~gn=H) gi L H gi+1 gi+1 UBi = (unchanged) UBi = UBi-1 - leak(gi=L) + leak(gi=H) LBi = LBi-1 - leak(gi=H) + leak(gi=L) LBi = (unchanged) Delay = (unchanged) Delay = (increased) Figure 4.4. Incremental leakage bound computation4.3.2.2 Incremental delay bound computation Similar to the leakage current bounds, a lower bound on the delay is com-puted assuming all unknown gates have low-Vt group assignments. Delay ischanged only when a high branch is taken in the traversal and is computedincrementally. We first compute the slack of the circuit for all circuit nodes atthe start of the tree traversal with all Vt assignments assumed to be low-Vt.When a group changes from a low to a high-Vt group assignment during thetraversal, the slack of that gate will be updated. However, the Vt change of thegate will affect not only the gate itself but also the delays of fanout gates dueto the slope change at the output of the changed gate. Since the slope at theoutput of the changed gate will become slower due to its high-Vt assignment,the delay of all fanout gates will increase, resulting an overall increased cir-cuit delay. Ignoring the effect of slope change on fanout gates will thereforeresult in the computation of an optimistic lower bound which ensures that theoptimal solution is not accidentally pruned. It also enables incremental delaycomputation, given that the gates are visited in topological ordering. As gatesare visited, the changed input slope, due to high-Vt assignments of a faningate, is processed to ensure that an exact delay bound is computed at the bot-tom of the gate tree.4.3.3 Heuristic Solution to Vt and State Assignment We propose two fast heuristics that can be applied to large circuits and thatproduce high quality solutions. The proposed heuristic are based on the exactmethod described in Section 4.3.2, and are discussed below.Heuristic 1 In this heuristic, the state and gate tree search is limited to only one down-ward traversal. Note that while only a single traversal of the state tree isperformed, at each node of the state tree the decision to follow the left or rightchild node is based on the computed bounds of the leakage using the gate tree.
  • 67Each downward traversal of the gate tree visits m nodes, where m is the num-ber of gates in the circuit. We perform exactly two such traversals at eachstate tree node, leading to a total run time complexity that is O(nm), where n isthe number of circuit inputs. Since the number of inputs is generally thoughtto grow approximately as the sqrt(m), the total complexity of this heuristic isO(msqrt(m)).Heuristic 2 In the second heuristic, the state tree is searched more extensively, subjectto a fixed run time constraint, while the gate tree search is kept to a singledownward traversal for each state tree node. Experimentally, it was found thatthe quality of the first bottom node reached in the gate tree search is near theoptimal Vt assignment. This is due to the fact that the gate tree always choosesthe high-Vt child in its downward traversal which tends to produce a highquality result. This is in contrast to the state tree, where choosing the correctchild during the downward traversal was found to be much more difficult.Therefore, the solution quality was found to improve most by searching thestate tree more extensively, subject to a run time constraint, while limiting thegate tree search to a single downward traversal.4.3.4 Vt assignment Control within Stacks We assume the ability to assign Vt on an individual basis within stacks oftransistors. Although it is generally possible to assign the Vt of each transistorin a stack individually, this may result in the need for increased spacingbetween the transistors in order not to violate design rules and ensure manu-facturability [26]. Hence, at times it may be desirable to restrict theassignment of Vt such that all transistors in a stack are uniform. In this case,less flexibility exists in the assignment of Vt, and hence the obtained trade-offin delay and leakage will degrade to some extent. In Section 4.5.1, we presentresults showing the impact on the leakage optimization when uniform stackassignments are enforced in the library.4.4 LEAKAGE REDUCTION METHOD FOR BOTH SUBTHRESHOLD AND GATE LEAKAGE CURRENT4.4.1 Leakage Reduction Approach The proposed leakage optimization method performs simultaneous assign-ment of standby mode state and high-Vt and thick-oxide transistors. The
  • 68proposed method is based on the key observation that given a known inputstate, a transistor need not be assigned both a high-Vt and a thick oxide. Thisis due to the fact that if a transistor that is OFF, gate leakage is significantlyreduced and hence the transistor only needs to be considered for high-Vtassignment. Conversely, a transistor that, given a particular input state, is ONmay exhibit significant Igate, but does not impact Isub. Hence, conducting tran-sistors only need to be considered for thick oxide assignment. If the input stateis unknown in standby mode, it cannot be predicted at design time which tran-sistors will be ON or OFF and therefore all or most transistors must beassigned to both high-Vt and thick-oxide in order to significantly reduce thetotal average leakage. However, given a known input state, we can avoidassignment of transistors to both high-Vt and thick oxide, thereby significantlyimproving the obtained leakage / delay trade-off. Furthermore, depending on the input state of a circuit, only a subset oftransistors needs to be considered for high-Vt or thick-oxide, as discussed inSection 4.3.1. For instance, in a stack of several transistors that are OFF, onlyone transistor needs to be assigned to high-Vt to effectively reduce the totalIsub. Similarly, Igate for transistors in a stack also has strong dependence ontheir position. If a conducting transistor is positioned above a non-conductingtransistor in a stack, its Vgs and Vgd will be small and gate leakage will bereduced. Hence, depending on the input state, only a small subset of all ONtransistors needs to be assigned thick-oxide and only a subset of all OFF tran-sistors need to be considered for high-Vt assignment. We illustrate the advantage of high-Vt and thick-oxide assignment with aknown input state for a 2-input NAND and NOR gate in Figure 4.5. In Figure4.5(a) a 2-input NOR gate is shown with input state 01. Since only PMOStransistors p2 is OFF in the pull-up stack, it is the only transistor that needs to p1 p1 p1 0 1 0 i1 i1 i1 p2 p2 p2 1 1 0 i2 i2 i2 n1 n2 n1 n2 n1 n2 (a ) (b ) (c ) p1 p2 p1 p2 0 1 i1 i2 H ig h -V t T ra n s is to r n1 n1 T h ic k O x id e n2 n2 T ra n s is to r 1 0 i2 i1 (d ) (e ) Figure 4.5. High Vt and thick oxide assignments at different input states
  • 69be set to high-Vt to reduce the subthreshold leakage of the gate. Similarly,only NMOS transistor n2 exhibits gate leakage and needs to be assigned thickoxide to reduce Igate. Hence only two out of four transistors are affected whilethe total leakage current is reduced by nearly the same amount as when alltransistors in the gate are set to high-Vt and thick oxide simultaneously. As aresult, the delay of the rising input transition at input i1 is unaffected by thehigh-Vt and thick-oxide assignments, while the other transitions are affectedonly moderately. In Figure 4.5(b), the worst-case input state for a NOR2 gate is shown,which is when both inputs are 1. In this case, both NMOS devices must beassigned to thick-oxide to reduce Igate, while at least one PMOS device is setto high-Vt. Depending on the delay requirements, the best input state is eitherthe state 01 shown in Figure 4.5(a), or the state 00, shown in Figure 4.5(c),which requires only two transistors to be set to high-Vt. Hence, it is clear thatthe input state significantly impacts the ability to effectively assign high-Vtand thick-oxides without degrading the performance of the circuit. This leadsto the need for a simultaneous optimization approach where both the inputstate and the high-Vt and thick-oxide assignments are considered simulta-neously under delay constraints. In addition to high-Vt and thick-oxide assignment, we also take advantageof the Igate dependence on input pin ordering to reduce leakage current [11].This is illustrated in Figure 4.5(d), for a 2-input NAND gate with input state01. In order to effectively reduce the leakage under this input state, NMOStransistor n1 must be assigned to high-Vt and NMOS transistor n2 must beassigned to thick-oxide. However, if input pins i1 and i2 are reordered, with i1positioned at the bottom of the stack, as shown in Figure 4.5(e), the Vgs andVgd voltage of NMOS transistor n1 will be reduced from Vdd to approximatelyone Vt drop. Hence, the gate leakage current of n1 will be substantiallyreduced and can be ignored. After reordering the input pins, it is necessary toonly set NMOS transistor n2 to high-Vt without further assignments of thick-oxide transistors. It should be noted that pin reordering will impact the delayof the circuit and hence some performance penalty might be incurred. How-ever, this penalty will be readily offset by the elimination of the thick-oxideassignment in the pull-down stack. In this paper, we therefore consider com-bined input state assignment with pin-reordering and Vt / Tox assignment.4.4.2 Cell Library Construction In order to perform simultaneous Vt, Tox and state assignment, it is neces-sary to develop a library where for each cell the necessary Vt and Tox versionare available. After such a library has been constructed, the process of assign-ing Vt and Tox assignments can be performed by simply swapping cells from
  • 70the library. Since different Vt and Tox variations do not alter the footprint of acell, the leakage optimization can be performed either before or after finalplacement and routing. For each gate and input state, a number of different Tox and Vt assignmentsis possible, providing different delay / leakage trade-off points. For the fastestand highest leakage trade-off point, all transistors are assigned to low-Vt andthin oxides, such as the NAND2 gate shown in Figure 4.6(a). On the otherhand, for the slowest and lowest leakage version of the cell all transistors con-tributing to leakage are assigned either high-Vt or thick oxide. For instance,for the NAND2 gate with input state 11, shown in Figure 4.6(b), all transistorsaffect the leakage current and both NMOS transistors are assigned thick Toxwhile both PMOS transistors are assigned high-Vt to obtain the minimumleakage / maximum delay trade-off point. In addition to the fastest version and minimum leakage version of the cell,a number of other intermediate trade-off points can be constructed for a cellby assigning only some of the transistors that contribute to leakage to high-Vtor thick-Tox. These cell versions would have lower leakage than the fastestcell version but would be faster than the lowest leakage version. It is clear thata large number of possible cell versions can be constructed if all possibletrade-off points are considered for each possible input state. While a larger setof cell versions provides the optimization algorithm with more flexibility, andhence a more optimal leakage result, it also increases the size of the library,which is undesirable. Therefore, we initially restrict our library to at most 4different trade-off points for each input state of a library cell, which are: 1) theminimum delay, shown in Figure 4.6(a), 2) minimum leakage, shown in Fig- tp1 tp2 tp1 tp2 tp1 tp2 A 1 A 1 A tn1 tn1 tn1 tn2 tn2 tn2 B 1 B 1 B (a) (b) (c) tp1 tp2 tp1 tp2 tp1 tp2 1 A 0 A 1 A tn1 tn1 tn1 tn2 tn2 tn2 1 B 0 B 0 B (d) (e) (f) Figure 4.6. Complete Vt-Tox versions of NAND2 gate
  • 71ure 4.6(b), 3) fast falling transition but slow rising transition, withintermediate leakage, shown in Figure 4.6(c), and 4) fast rising transition butslow falling transition with intermediate leakage, shown in Figure 4.6(d).Although other possible trade-off points could be considered, we empiricallyfound that these four points yield good optimization results and provide a sys-tematic approach for constructing all versions of a cell. In principle, using four possible trade-off points for each input combina-tion could result in as many as 16 (4x4) cell versions for a 2 input gate.However, in practice, many of the cell versions are shared between differentinput states. Also, in some cases not all 4 trade-off points are realizable andhence the total number of cell versions is significantly less. We illustrate thisfor the NAND2 gate for input state 00. The fastest cell version is again shownin Figure 4.6(a) and is shared for all input combinations, and the minimumleakage version is shown in Figure 4.6(e). Note that only one transistor needsto be set to high-Vt to achieve minimum leakage for this input state. Thisresults from the fact that PMOS devices have negligible gate leakage in thetarget technology and only one transistor in a stack needs to be set to high-Vtto reduce the leakage through the entire stack. Hence, for the input state 00,only two trade-off points are needed and only one additional cell version isadded to the library. Input state 10 again requires the assignment of only a single transistor tohigh-Vt for the minimum leakage version, as shown in Figure 4.6(f). This isdue to the fact that the gate leakage through the top NMOS transistor n1 isnegligible since its Vgs and Vgd is reduced to approximately one Vt drop. Onlytwo trade-off points are therefore required for this input state and both ver-sions are shared with the 00 state. Finally, if the 01 state occurs in the circuit,the optimization will automatically perform input pin swapping for all but thefastest trade-off point, thereby resulting in no additional cell version. TheNAND2 gate therefore requires a total of 5 cell versions to provide up to 4trade-off points for each input state. In Table 4.2, we show the delay / leakage Table 4.2. Trade-offs for different Vt-Tox versions of NAND2 gate Total Normalized Normalized leakage rise delay fall delay State Cell current [nA] pin A pin B pin A pin B Minimum delay (a) 270.4 1.00 1.00 1.00 1.00 Fast rise delay (d) 109.1 1.00 1.36 1.27 1.27 11 Fast fall delay (c) 91.4 1.36 1.36 1.00 1.00 Minimum leakage (b) 19.5 1.36 1.37 1.27 1.27 Minimum delay (a) 41.2 1.00 1.00 1.00 1.00 00 Minimum leakage (e) 14.0 1.00 1.00 1.12 1.16 Minimum delay (a) 91.8 1.00 1.00 1.00 1.00 10 Minimum leakage (f) 13.3 1.00 1.00 1.12 1.16
  • 72trade-offs obtained for each input state using the described approach for theNAND2 gate. The same process can be applied to each cell in the library to construct thefull set of cell versions for the leakage characterization method. Table 4.3,shows the number of cell version required for several common gates. Notethat the number of cell version is higher for NOR gates than NAND gates.Since for a library the total number of cells would increase significantly, wealso explored reducing the number of cells by allowing only two trade-offpoints for each cell (minimum delay, and minimum leakage), instead of 4trade-off points. In this case, the number of cells for the NAND2 gate reducesto only 3 versions. The number of cell version required for two trade-offpoints for different cell types is shown in Table 4.3, column 3. In column 4,we add one more cell library version - two trade-off points with reduced num-ber of cells. In order to minimize the number of needed library cells, one ortwo cells of NOR2 or NOR3, respectively, are removed from library withsmall degradation of leakage/delay trade-off. Therefore, all gates have onlythree cells in this option. In Section 4.5.2 we compare the final leakage resultsusing the full library with 4 trade-off points, the reduced library with only twotrade-off points, and minimum number of cell library with two trade-offpoints. Finally, we consider Vt and Tox assignment control within stacks similar tothe discussion for Vt stack control in Section 4.3.4. However, Tox assignmentdiffers from Vt assignment in that the assignment of Tox to transistors in astack is already uniform due to the use of pin-swapping. This is evident fromthe 5 added cell versions for the NAND2 in Figure 4.6, and can be easilyshown to be true for all cell versions generated under the proposed approach.This is a significant advantage since spacing design rules for different Toxassignments are expected to be more severe that those for spacing betweendifferent Vt assignments [26]. However, the Vt assignment is not always uni-form as shown in Figure 4.6(e), where only a single transistor in a stack isassigned to high-Vt. In the event that a uniform stack is required, both transis-tors in the stack need to be set to high-Vt, resulting in a slightly worsened Table 4.3. The number of needed library cells 2 trade-off points 4 trade-off points 2 trade-off points with reduced number of cells Inverter 5 3 3 NAND2 5 3 3 NAND3 5 3 3 NOR2 8 4 3 NOR3 9 5 3
  • 73 Gate tree g1 g2 g2 State tree s1 0 1 gm s2 s2 0 1 0 1 Sol sn sn 0 1 0 1 Sol Figure 4.7. State tree with gate tree at each nodedelay / leakage trade-off. Leakage current comparison results between indi-vidual vs. uniform stack assignment control will be shown in Section Optimization - Approach and Heuristics In this section, we present an exact solution and two heuristics to the prob-lem of finding a simultaneous input state, high-Vt and thick-Tox assignmentsfor a circuit under delay constraints. As mentioned, the leakage minimizationproblem can be formulated as a integer optimization problem under delayconstraints. The size of the input state space is 2n, where n is the number ofcircuit inputs. As discussed in Section 4.4.2, for each input state assignment,there are up to four possible Vt-Tox assignments for each gate. Note that whilethe total number of cell versions can be larger than 4, only 4 of them need tobe considered for each specific input state. For instance, for the NAND2 gatein Figure 4.6, only versions (a)-(d) are considered for a 11 input state. There-fore, the total number of possible Vt-Tox assignments is 4m, where m is thenumber of gates in the circuit and the total size of the search space is 2n+2m. In order to find an exact solution to the problem, we extend the branch-and-bound method with Section 4.3.2. The branch and bound algorithm forVt-Tox and state assignment uses two interdependent search trees: state treeand gate tree. The state tree is searched to determine the input state of the cir-cuit and the gate tree is searched to determine the Vt-Tox assignment of thecircuit, as shown in Figure 4.7. The only difference from Section 4.3.2 is thegate tree. Each node in a particular gate tree corresponds to a gate in the cir-cuit. Since there are four possible Vt-Tox assignments for a gate, each node ofthe gate tree has four edges: minimum delay, minimum leakage, fast fall delaywith intermediate leakage, and fast rise delay with intermediate leakage. Theexponential nature of the problem makes it impossible to obtain an exact solu-
  • 74tion for substantial circuits, such as Isub minimization approach in Section4.3.2. Therefore, we also use the two heuristics discussed in Section RESULTS4.5.1 Subthreshold Leakage Reduction The proposed methods for simultaneous state and Vt assignment weretested on the ISCAS benchmark circuits [27] and a 64-bit ALU circuit, syn-thesized using a 0.18 m industrial library with Synopsys. This technology hasa difference of 14X (10X) in Isub and 16% (15%) in delay between low-Vt andhigh-Vt NMOS (PMOS) devices. The leakage current for each Vt version of acell was computed using SPICE simulation and stored in precharacterizedtables. Delay computation was performed based on the Synopsys table delaymodel and was verified to match with Synopsys timing analysis delay reports.In addition to the proposed methods, traditional methods using only state or Vtassignment were also implemented for comparison. The state-only assign-ment was implemented using the approach discussed in [25] while for Vt-onlyassignment a method similar to the sensitivity-based approach of [17] wasused. Table 4.4 compares the leakage results obtained by the three proposedheuristics for three delay constraints to the average leakage computed using10,000 random input vectors. The columns marked 0%, 5%, and 10% refer toleakage minimization results when the delay constraints were set at 0%, 5%,and 10% respectively, of the full delay range between all low-Vt and all high-Vt circuit delay, as illustrated in Figure 4.8. The 0% column is therefore the Table 4.4. Leakage current comparison between heuristics Minimized leakage current [nA] (reduction factor: vs. average leakage current) 0% in low Vt/high Vt delay range 5% in low Vt/high Vt delay range 10% in low Vt/high Vt delay range Avg. Ileak by random Heuristic 1 Heuristic 2 Heuristic 1 Heuristic 2 Heuristic 1 Heuristic 2 (10K)vectors Ileak X Time Ileak X Ileak X Time Ileak X Ileak X Time Ileak X C432 32.9 7.7 4.3 1 4.3 7.7 4.9 6.7 1 3.6 9.2 4.7 7.0 1 3.6 9.1 C499 94.0 13.2 7.1 3 11.3 8.3 13.1 7.2 2 11.6 8.1 9.7 9.6 2 9.7 9.6 C880 73.4 9.7 7.5 4 8.9 8.3 8.9 8.2 3 8.3 8.8 8.9 8.3 4 8.3 8.8 C1355 85.1 19.0 4.5 3 12.7 6.7 14.6 5.8 3 11.7 7.3 12.0 7.1 3 11.0 7.7 C1908 82.8 19.0 4.3 2 15.1 5.5 15.5 5.3 2 12.2 6.8 13.4 6.2 2 10.3 8.0 C2670 162.5 12.7 12.8 58 12.5 13.0 12.7 12.8 55 12.4 13.1 14.3 11.3 55 12.2 13.3 C3540 173.1 20.1 8.6 10 16.4 10.6 20.5 8.4 10 14.6 11.8 17.4 10.0 9 14.5 11.9 C5315 309.1 26.4 11.7 169 25.9 11.9 27.5 11.2 164 25.2 12.3 28.5 10.9 165 25.2 12.2 C6288 451.5 157.5 2.9 47 153.9 2.9 145.5 3.1 44 141.4 3.2 135.8 3.3 43 128.4 3.5 C7552 385.8 31.0 12.4 330 30.6 12.6 30.8 12.5 330 30.1 12.8 30.7 12.6 328 29.6 13.0 alu64 332.3 46.0 7.2 405 43.6 7.6 47.2 7.0 408 44.5 7.5 43.0 7.7 406 42.0 7.9 AVG 7.6 8.6 8.0 9.2 8.5 9.6
  • 75 Delay with all low-Vt Delay with all high-Vt 0% 100% 5% 10% 50% Figure 4.8. Delay point from all low-Vt to all high-Vt rangemost stringently constrained optimization as it corresponds to the best obtain-able delay for the circuit (no performance penalty). Note that a simplereplacement of all low-Vt devices with all high-Vt ones would yield a ~20%circuit delay increase. Thus, when interpreting the results in this section, a10% delay point indicates that the circuit after Vt assignment has a delay thatis approximately 2% larger than the original fastest implementation. Since theaverage leakage current with 10,000 random input vectors is computed withall low-Vt transistors, it also corresponds to a 0% delay criteria. Runtimes forheuristic 1 are given in Table 4.4 in seconds. Heuristic 2 was limited to a runt-ime of 1800 seconds (30 minutes). We report the reduction factor relative tothe average leakage current over the 10,000 random vectors. Heuristic 2 has~10% lower leakage results than heuristic 1 at 5% delay point across thebenchmark circuits. However, heuristic 2 has a 4-5X runtime overhead forlarge circuits (~1000X for small circuits) over heuristic 1. In Table 4.5, we compare the proposed approach with traditional tech-niques, including state-only and Vt-only assignment methods. The state-only Table 4.5. Leakage current comparison with traditional techniques Circuits Minimized leakage current [nA] Avg. Vt only & proposed heuristic (reduction factor: vs. average leakage current) Number of State-only Ileak by 0% in the delay range 5% in the delay range 10% in the delay range assignment random (10K) Vt-only Heuristic 1 Vt-only Heuristic 1 Vt-only Heuristic 1 Input Gate vectors Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X C432 36 177 32.9 26.3 1.25 30.8 1.1 7.7 4.3 29.5 1.1 4.9 6.7 29.2 1.1 4.7 7.0 C499 41 519 94.0 86.1 1.09 85.0 1.1 13.2 7.1 57.2 1.6 13.1 7.2 40.4 2.3 9.7 9.6 C880 60 364 73.4 63.7 1.15 64.6 1.1 9.7 7.5 63.9 1.1 8.9 8.2 20.2 3.6 8.9 8.3 C1355 41 528 85.1 81.4 1.04 94.0 0.9 19.0 4.5 65.1 1.3 14.6 5.8 53.4 1.6 12.0 7.1 C1908 33 432 82.8 74.6 1.11 67.0 1.2 19.0 4.3 46.5 1.8 15.5 5.3 30.3 2.7 13.4 6.2 C2670 233 825 162.5 146.2 1.11 44.7 3.6 12.7 12.8 39.7 4.1 12.7 12.8 27.8 5.8 14.3 11.3 C3540 50 940 173.1 155.7 1.11 161.9 1.1 20.1 8.6 148.4 1.2 20.5 8.4 82.4 2.1 17.4 10.0 C5315 178 1627 309.1 283.1 1.09 290.6 1.1 26.4 11.7 289.7 1.1 27.5 11.2 108.3 2.9 28.5 10.9 C6288 32 2470 451.5 412.4 1.09 417.0 1.1 157.5 2.9 259.5 1.7 145.5 3.1 233.0 1.9 135.8 3.3 C7552 207 1994 385.8 352.3 1.10 360.2 1.1 31.0 12.4 353.5 1.1 30.8 12.5 350.9 1.1 30.7 12.6 alu64 131 1803 332.3 294.5 1.13 312.8 1.1 46.0 7.2 288.5 1.2 47.2 7.0 230.1 1.4 43.0 7.7 Avg. 1.12 1.3 7.6 1.6 8.0 2.4 8.5
  • 76assignment method was limited to a runtime of 1800 seconds (30 minutes).The results demonstrate that substantial improvement in standby leakage cur-rent can be obtained using the proposed methods, with an averageimprovement of ~80% (5-6X) for the 0% and 5% delay constraints over Vt-only assignment. Table 4.6 compares leakage current results for both individual and uni-form stack control. Since uniform stack control degrades the delay/leakagetrade-off as discussed in Section 4.3.4, the results for uniform stack assign-ment exhibit less leakage reduction than those of individual stack control. It isinteresting to note, however, that the leakage current degradation by movingto a less fine-grained threshold voltage assignment scheme is not overly largeimplying that even with manufacturing constraints, the proposed techniqueprovides significant leakage savings. Finally, Figure 4.9 plots the leakage results for the proposed method andthe two traditional methods as a function of the delay constraint for circuitc6288. The optimization was performed for a range of delay constraints. Theproposed method provides its largest improvements at tight delay constraints.This is due to the fact that, as the delay constraint becomes looser, more tran-sistors can be set to high-Vt in both approaches, and the relative advantage ofthe proposed approach reduces. However, leakage reduction is most challeng-ing under tight performance constraints at which the proposed techniqueholds promise. Table 4.6. Leakage current comparison between individual and uniform stack control. Minimized leakage current [nA] 5% in low Vt/high Vt delay range (reduction factor: vs. average leakage current) Average Heuristic 1 Vt-only assignment Ileak by Individual control Uniform control random (10K)vector Ileak X Ileak X Ileak X C432 32.9 29.5 1.1 4.9 6.7 6.8 4.8 C499 94.0 57.2 1.6 13.1 7.2 12.5 7.5 C880 73.4 63.9 1.1 8.9 8.2 9.1 8.1 C1355 85.1 65.1 1.3 14.6 5.8 23.7 3.6 C1908 82.8 46.5 1.8 15.5 5.3 15.7 5.3 C2670 162.5 39.7 4.1 12.7 12.8 12.9 12.6 C3540 173.1 148.4 1.2 20.5 8.4 24.1 7.2 C5315 309.1 289.7 1.1 27.5 11.2 28.5 10.9 C6288 451.5 259.5 1.7 145.5 3.1 163.1 2.8 C7552 385.8 353.5 1.1 30.8 12.5 31.3 12.3 alu64 332.3 288.5 1.2 47.2 7.0 44.6 7.5 Avg. 1.6 8.0 7.5
  • 77 450 Average Current with Low-Vt 400 State Assignment Only with Low-Vt Dual-Vt Assignment only 350 Our proposed method - Heuristic 1 Total Leakage Current [nA] State Assignment Only with High-Vt 300 250 200 150 100 50 0 0 10 20 30 40 50 60 70 80 90 100 Delay Point from All Low-V t to All High-Vt Range [%] Figure 4.9. Leakage current comparison for c62884.5.2 Leakage Reduction for both Subthreshold and Gate Leakage The proposed methods for simultaneous state, Vt and Tox assignment wereimplemented on a number of benchmark circuits [27] synthesized using alibrary based on a predictive 65nm process [20]. In this technology, the differ-ence in Igate for the thick-oxide NMOS devices vs. the thin-oxide device is11X, whereas Isub is reduced by 17.8X (16.7X) when replacing a low-VtNMOS (PMOS) device with a high-Vt version. Table 4.7 shows relative leak-age and delay values at the four possible Vt and Tox assignments for NMOSdevices in this technology. A comparison of our first and second heuristicsalong with average leakage computed using 10,000 random input vectors isshown in Table 4.8. The total leakage current value is given in A and runt-ime is given in seconds. In heuristic 2, we set the runtime limit as 1800 Table 4.7. Comparison of leakage and delay between four possible Vt-Tox assignment for NMOS Assignment Normalized values Leakage Vt Oxide thickness Delay Isub Forward Igate Reverse Igate Low Thin 1.00 0.41 0.22 1.00 High Thin 0.06 0.31 0.22 1.33 Low Thick 0.73 0.04 0.00 1.26 High Thick 0.05 0.03 0.02 1.69
  • 78 Table 4.8. Leakage current comparison between heuristics with 4-option, individual stack control library Average 0% in the best-worst delay range 5% in the best-worst delay range 10% in the best-worst delay range Ileak by random Heu1 Heu2 Heu1 Heu2 Heu1 Heu2 (10K) vectors Ileak X Time Ileak X Ileak X Time Ileak X Ileak X Time Ileak X c432 24.5 8.2 3.0 3 5.4 4.6 7.7 3.2 2 3.2 7.6 5.5 4.5 2 3.0 8.2 c499 65.8 32.2 2.0 7 31.1 2.1 26.1 2.5 7 24.6 2.7 22.7 2.9 6 20.8 3.2 c880 50.1 10.3 4.9 8 9.2 5.5 8.5 5.9 7 8.3 6.1 8.5 5.9 7 7.0 7.1 c1355 70.8 20.4 3.5 8 20.4 3.5 15.8 4.5 6 13.1 5.4 9.9 7.1 6 9.9 7.1 c1908 56.7 17.4 3.3 5 16.9 3.4 14.8 3.8 4 13.6 4.2 13.2 4.3 5 10.5 5.4 c2670 104.7 14.9 7.0 82 14.7 7.1 12.3 8.5 78 12.2 8.6 13.5 7.8 78 11.3 9.3 c3540 128.5 27.7 4.6 20 23.7 5.4 22.1 5.8 18 19.9 6.4 18.6 6.9 17 17.4 7.4 c5315 221.2 36.6 6.0 219 35.9 6.2 30.0 7.4 213 30.0 7.4 28.4 7.8 202 27.6 8.0 c6288 346.8 153.6 2.3 75 146.0 2.4 112.2 3.1 64 101.4 3.4 84.1 4.1 59 75.6 4.6 c7552 270.0 34.9 7.7 410 33.4 8.1 32.2 8.4 404 31.8 8.5 30.3 8.9 399 30.2 8.9 alu64 260.0 48.7 5.3 468 46.8 5.6 43.4 6.0 464 41.6 6.3 34.3 7.6 458 33.1 7.9 AVG 4.5 4.9 5.4 6.0 6.2 7.0seconds (30 minutes). The average leakage computed using the random vec-tors can be used to approximate the standby mode leakage if state assignmentas well as dual-Vt and dual-Tox techniques were not employed. Again, thedelay range points used in all results are defined by a percentage of the maxi-mum possible delay that is associated with moving from an all low-Vt andthin-oxide design to an all high-Vt and thick-oxide implementation. Note thata simple replacement of all fast devices with their slowest counterparts wouldyield a ~70% circuit delay increase. Thus, when interpreting the results in thissection, a 5% delay point indicates that the circuit after Vt and Tox assignmenthas a delay that is approximately 4% larger than the original fastestimplementation. As shown in Table 4.8, heuristic 2 generally provides somewhat betterresults but at much greater runtimes. On average, heuristic 2 provides ~10%lower leakage current than heuristic 1 across these benchmarks at the 5%delay point, similar to the results in Section 4.5.1. The improvement of thetwo proposed heuristics compared to the average leakage without state, Vt orTox assignment is dramatic and approaches 7X at the 10% delay point in thebest-worst delay range. More aggressively, with just a 5% delay penalty thereduction in total standby leakage is 5.3-6X with a maximum improvement of8.6X for heuristic 2 in circuit c2670. In Table 4.9 we compare our results to other standby mode techniques,including state assignment alone and simultaneous state and Vt assignment (asin the previous section). The total leakage current value is given in A. Again,we report the reduction factor in relation to the average leakage current with10,000 random vectors for consistency. We first point out that state assign-
  • 79 Table 4.9. Leakage current comparison with 4-option, individual stack control library Average State 0% in the delay range 5% in the delay range 10% in the delay range Ileak by Assignment random Only Vt & State Heu1 Vt & State Heu1 Vt & State Heu1 (10K) vectors Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X c432 24.5 22.7 1.08 13.3 1.8 8.2 3.0 12.5 2.0 7.7 3.2 12.7 1.9 5.5 4.5 c499 65.8 63.9 1.03 41.9 1.6 32.2 2.0 35.7 1.8 26.1 2.5 32.2 2.0 22.7 2.9 c880 50.1 46.0 1.09 18.9 2.6 10.3 4.9 17.5 2.9 8.5 5.9 16.9 3.0 8.5 5.9 c1355 70.8 67.4 1.05 39.9 1.8 20.4 3.5 33.0 2.1 15.8 4.5 29.8 2.4 9.9 7.1 c1908 56.7 54.8 1.04 27.6 2.1 17.4 3.3 25.8 2.2 14.8 3.8 22.9 2.5 13.2 4.3 c2670 104.7 101.4 1.03 33.3 3.1 14.9 7.0 32.7 3.2 12.3 8.5 31.9 3.3 13.5 7.8 c3540 128.5 121.8 1.05 54.5 2.4 27.7 4.6 51.5 2.5 22.1 5.8 48.5 2.7 18.6 6.9 c5315 221.2 215.1 1.03 81.2 2.7 36.6 6.0 77.1 2.9 30.0 7.4 73.7 3.0 28.4 7.8 c6288 346.8 306.7 1.13 209.3 1.7 153.6 2.3 180.4 1.9 112.2 3.1 153.7 2.3 84.1 4.1 c7552 270.0 262.6 1.03 88.9 3.0 34.9 7.7 86.6 3.1 32.2 8.4 86.1 3.1 30.3 8.9 alu64 260.0 237.2 1.10 90.7 2.9 48.7 5.3 86.1 3.0 43.4 6.0 81.1 3.2 34.3 7.6 AVG 1.06 2.3 4.5 2.5 5.4 2.7 6.2ment alone, which we accomplish by searching the state tree only, achievesvery little improvement in standby mode leakage; about 6%. By adding Vtassignment, the algorithm of the first proposed method shows an averagereduction of 58% beyond state assignment alone at a 5% delay point. The fullVt, Tox, and state assignment approach provides an additional 53% reductionin current beyond state and Vt assignment for the 5% delay point. Table 4.10 provides a comparison of results using the various cell libraryoptions; 4 and 2 trade-off points with individual stack control, and also withuniform stacks. The main result in Table 4.10 is that there is very little leak- Table 4.10. Leakage current comparison between cell library options (current unit: A) 5% in the best-worst delay range Average Individual stack control Uniform stack control Ileak by random 2-option 2-option (10K) 4-option 2-option 3 cell versions 4-option 2-option 3 cell versions vectors only only Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X c432 24.5 7.7 3.2 7.4 3.3 7.1 3.4 7.3 3.4 7.9 3.1 8.6 2.8 c499 65.8 26.1 2.5 26.7 2.5 27.8 2.4 26.0 2.5 28.0 2.3 28.9 2.3 c880 50.1 8.5 5.9 9.7 5.2 8.0 6.3 10.0 5.0 10.7 4.7 10.8 4.6 c1355 70.8 15.8 4.5 16.2 4.4 14.1 5.0 23.4 3.0 25.2 2.8 23.9 3.0 c1908 56.7 14.8 3.8 14.9 3.8 14.3 4.0 15.9 3.6 15.3 3.7 16.8 3.4 c2670 104.7 12.3 8.5 12.1 8.7 12.4 8.4 16.1 6.5 15.4 6.8 16.5 6.3 c3540 128.5 22.1 5.8 24.2 5.3 25.3 5.1 27.1 4.7 25.8 5.0 29.2 4.4 c5315 221.2 30.0 7.4 30.9 7.2 30.7 7.2 32.1 6.9 32.9 6.7 33.8 6.6 c6288 346.8 112.2 3.1 114.2 3.0 114.2 3.0 134.0 2.6 147.8 2.3 145.4 2.4 c7552 270.0 32.2 8.4 31.4 8.6 30.6 8.8 31.8 8.5 31.1 8.7 31.1 8.7 alu64 260.0 43.4 6.0 44.0 5.9 43.2 6.0 42.0 6.2 47.0 5.5 46.1 5.6 AVG 5.4 5.3 5.4 4.8 4.7 4.6
  • 80age current penalty when moving from a full 4-option library to a simpler 2-option library. There are several cases where the smaller library outperformsthe larger library due to the heuristic nature of the algorithm used (heuristic 1is used in this table). Since the library size required in the 2-option scenario isroughly half that of 4-option, we conclude that the use of 2-option represents avery good trade-off between library complexity and potential leakage reduc-tion. Moreover we can see that the simplest cell library of 2-option with areduced number of cells provides good leakage reduction results. In general, areduced number of cells degrades the leakage/delay trade-off as discussed inSection 4.4.2. However we find that only complex, and infrequently usedcells, such as 3-input NORs require appreciable reductions in cell variantswhich limits the impact on total leakage reduction. Therefore, very good leak-age current minimization can be obtained even with libraries with 3 cellversions for each cell. Also, the restriction that each stack of transistors mustuse the same Vt and Tox is shown in Table 4.10 to have only a minor impact onleakage. For instance, the uniform stack 4-option case shows a 10.6% averagepower increase compared to the individual stack 4-option case; this still repre-sents a nearly 5X reduction in standby leakage compared to the average case.Note that library complexity is not reduced in moving from individual tostack-based control; such a change would be dictated by manufacturing issuesas well as the trade-off between standby power (lower for individual control)and cell area (expected to be slightly lower for stack-based control). Finally, Figure 4.10 plots the leakage current results for the proposedmethod and traditional methods as a function of the delay constraint for cir- 350 Average Current with Low-Vt/Thin-T ox State Assignment Only with Low-V t/Thin-T ox 300 Dual-Vt & State Assignment Our proposed method - Heuristic 1 Total Leakage Current [uA] 250 State Assignment Only with High-V t/Thick-T ox 200 150 100 50 0 0 10 20 30 40 50 60 70 80 90 100 Delay Point from the best to the worst range [%] Figure 4.10. Leakage current comparison for c6288
  • 81cuit c6288. Here, a 100% delay point implies a complete replacement of low-Vt and thin-oxide devices with high-Vt and thick-oxide. This is clearly thelowest leakage solution but is also very slow. The key point in Figure 4.10 isthat the proposed approaches (heuristic 2 results are not shown but are nearlyidentical to heuristic 1) provide substantial improvement beyond the averageleakage or the use of state assignment alone and that these gains are achiev-able with very small and even zero delay penalties. The rapid saturation of thegains as the delay point increases beyond 10% implies that the new approachis best suited for achieving low-leakage standby states with very little perfor-mance overhead (e.g., 5% or even less). Note that the leakage currentachieved by our proposed method does not converge to that by state assign-ment using all high-Vt and thick-oxide devices. The reason is that the selectedlibrary cells include only a limited number of thick-oxide assignments inorder to simplify the library. Many additional library cells would be needed toachieve convergence to the minimal leakage solution; instead the bulk of thisleakage savings can be achieved with very little performance penalty.4.6 CONCLUSIONS In this paper, we propose new approaches for standby leakage currentminimization under delay constraints. Our approaches use simultaneous stateassignment and Vt or Vt / Tox assignment. Efficient methods for computing thesimultaneous state and Vt or Vt / Tox assignments leading to the minimumstandby mode leakage current were presented. The proposed methods wereimplemented and tested on a set of synthesized benchmark circuits. Using thenew state and Vt assignment technique demonstrates 6X lower leakage thanprevious Vt-only assignment approaches and 5X lower than state assignmentalone (at 5% delay point). In cases where gate leakage is prominent, as in90nm CMOS technologies, these improvements are increased by an addi-tional factor of 2 using state and Vt / Tox assignment. We also investigate theleakage/complexity trade-off for various cell library configurations and dem-onstrate that results are still very good even when only 2 additional variantsare used for each cell type.Acknowledgement The authors would like to thank Harmander Deogun for his work in leak-age current model. The work has been supported by NSF, SRC, GSRC/DARPA, IBM, and Intel.
  • 82References[1] S. Mutoh, T. Douseki, Y. Matsuya, T. Aoki, S. Shigematsu and J. Yamada, “1-V power supply high-speed digital circuit technology with multithreshold voltage CMOS,” IEEE Journal of Solid-State Circuits, vol. 30, pp. 847-854, Aug. 1995.[2] J. Kao, A. Chandrakasan, and D. Antoniadis, “Transistor sizing issues and tool for multi-threshold CMOS technology,” Proc. Design Automation Conference, pp. 409-414, 1997.[3] S. Shigematsu, S. Mutoh, Y. Matsuya, Y. Tanabe and J. Yamada, “A 1-V high- speed MTCMOS circuit scheme for power-down application circuits,” IEEE Journal of Solid-State Circuits, vol. 32, pp. 861-869, June 1997.[4] H. Kawaguchi, K. Nose and T. Sakurai, “A super cut-off CMOS (SCCMOS) scheme for 0.5V supply voltage with picoampere standby current,” IEEE Journal of Solid-State Circuits, vol. 35, pp. 1498-1501, October 2000.[5] R. X. Gu and M. I. Elmasry, “Power dissipation analysis and optimization of deep submicron CMOS digital circuits,” IEEE Journal on Solid-State Circuits, vol. 31, no. 5, pp. 707-713, May 1996.[6] Z. Chen, M. C. Johnson, L. Wei and K. Roy, “Estimation of standby leakage power in CMOS circuit considering accurate modeling of transistor stacks,” Proc. International Symposium on Low Power Electronics Design, pp. 239-244, 1998.[7] J. Halter and F. Najm, “A gate-level leakage power reduction method for ultra- low-power CMOS circuits,” Proc. CICC, pp. 475-478, 1997.[8] V. De, Y. Ye, A. Keshavarzi, S. Narendra, J. Kao, D. Somasekhar, R. Nair and S. Borkar, “Techniques for leakage power reduction,” in Design of High-Perfor- mance Microprocessor Circuits, New York: IEEE Press, 2001.[9] M.C. Johnson, D. Somasekhar and K. Roy, “Models and algorithms for bounds on leakage in CMOS circuits,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 18, pp. 714-725, June 1999.[10] A. Fadi, S. Hassoun, K. A. Sakallaha and D. Blaauw, “Robust SAT-based search algorithm for leakage power reduction,” Proc. International Workshop on Power and Timing Modeling, Optimization and Simulation, 2002.[11] D. Lee, W. Kwong, D. Blaauw and D. Sylvester, “Analysis and minimization techniques for total leakage considering gate oxide leakage,” Proc. Design Auto- mation Conference, pp. 175-180, 2003.[12] R.S. Guindi and F.N. Najm, “Design techniques for gate-leakage reduction in CMOS circuits,” Proc. ISQED, pp.61-65, 2003.[13] F. Hamzaoglu and M.R. Stan, “Circuit-level techniques to control gate leakage for sub-100nm CMOS,” Proc. International Symposium on Low Power Electron- ics and Design, pp. 60-63, 2002.
  • 83[14] T. Inukai, M. Takamiya, K. Nose, H. Kawaguchi, T. Hiramoto and T. Sakurai, “Boosted Gate MOS (BGMOS): Device/circuit cooperation scheme to achieve leakage-free giga-scale integration,” Proc. Custom Integrated Circuit Confer- ence, pp. 409-412, 2000.[15] Q. Wang and S.B.K. Vrudhula, “Static power optimization of deep submicron CMOS circuits for dual Vt technology,” International Conference on Computer- Aided Design, pp. 490-496, 1998.[16] L. Wei, Z. Chen, M. C. Johnson, K. Roy and V. De, “Design and optimization of low voltage high performance dual threshold CMOS circuits,” Proc. Design Automation Conference, pp. 489-494, 1998.[17] S. Sirichotiyakul, T. Edwards, C. Oh, R. Panda and D. Blaauw, “Duet: an accu- rate leakage estimation and optimization tool for dual Vt circuits,” IEEE Transac- tions on Very Large Scale Integration (VLSI) Systems, vol. 10, pp. 79-90, April 2002.[18] M. Ketkar and S. Sapatnekar, “Standby power optimization via transistor sizing and dual threshold voltage assignment,” Proc. ICCAD, 2002, pp. 375-378.[19] S. Stiffler, “Optimizing performance and power for 130nm and beyond,” IBM Technology Group New England Forum, 2003.[20] International Technology Roadmap for Semiconductors, 2002.[21] N. Yang, W. K. Henson, and J. J. Wortman, “A comparative study of gate direct tunneling and drain leakage currents in N-MOSFETs with sub-2nm gate oxides,” IEEE Trans. Electron Devices, vol. 47, pp. 1636-1644, Aug. 2000.[22] B. Yu, H. Wang, C. Riccobene, Q. Xiang and M.-R. Lin “Limits of gate oxide scaling in nano-transistors,” Proc. Symposium on VLSI Tech., pp. 90-91, 2000.[23] Y.-C. Yeo, Q. Lu, W.-C. Lee, T.-J. King, C. Hu, X. Wang, X. Guo and T. P. Ma, “Direct tunneling gate leakage current in transistors with ultra thin silicon nitride gate dielectric,” IEEE Electron Device Letters, vol. 21, pp. 540-542, Nov. 2000.[24] Q. Xiang, J. Jeon, P. Sachdey, B. Yu, K. C. Saraswat and M.-R. Lin, “Very high performance 40nm CMOS with ultra-thin nitride/oxynitride stack gate dielectric and pre-doped dual poly-Si gate electrodes,” Proc. International Electron Devices Meeting, pp. 860-862, 2000.[25] H. Kriplani, F. N. Najm and I. N. Hajj, “Pattern independent maximum current estimation in power and ground buses of CMOS VLSI circuits: algorithms, sig- nal correlations, and their resolution,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 14, pp. 998-1012, Aug. 1995.[26] Ruchir Puri, IBM T.J. Watson Research, personal communication.[27] F. Brglez and H. Fujiwara, “A Neutral Netlist of 10 Combinatorial Benchmark Circuits”, Proc. ISCAS, 1985, pp.695-698.
  • 84Chapter 5ENERGY-EFFICIENT SHARED MEMORYARCHITECTURES FOR MULTI-PROCESSORSYSTEMS-ON-CHIPKimish Patel1, Alberto Macii1 and Massimo Poncino21 Politecnico di Torino; 2 Universit` di Verona aAbstract Most current multi-processor systems-on-chip (MPSoC) platforms do rely on a shared-memory architectural paradigm. The shared memory, typically used for storage of shared data, is a significant performance bottleneck because it re- quires explicit synchronization of memory accesses which can potentially occur in parallel. Multi-port memories are a widely-used solution to this problem; they allow these potentially parallel accesses to occur simultaneously. However, they are not very energy-efficient, since their performance improvement comes at an increased energy cost per access. We propose an energy-efficient architecture for the shared memory that can be used as an alternative to multi-port mem- ories, and combines their performance advantage with a much smaller energy cost. The proposed scheme is based on the application-driven partitioning of the shared address space into a multi-bank architecture. This optimization can be used to quickly explore different power-performance tradeoffs, thanks to simple analytical models of performance and energy. Experiments on a set of paral- lel benchmarks show energy-delay product (EDP) savings of 50% on average, measured on a set of standard parallel benchmarks.Keywords: Multi-Processor Systems, Shared Memory, Systems-on-Chip.5.1 INTRODUCTION Modern design paradigms for MPSoCs are pushing towards architectureswhich are fully distributed and that work as general networks, based on a mod-ular layered architecture, and that are able to support non-deterministic com-munications. Such architectures, called Networks-on-Chips (NoCs) [1], havebeen devised as an answer to the scaling of SoC complexity, especially in termsof the increased number of hosted processing elements, and of the decreasedreliability of the communication medium.
  • 85 In spite of these scalability challenges, most current SoCs are still based on ashared-medium architecture, and, consequently on a shared-memory paradigm.One reason for this slow migration to more complex architectures is cost. Sharedon-chip buses represent a convenient, low-overhead interconnection, and theydo not require special handling during the physical design flow. Another reasonis a consequence of the limited support provided by system software for sucharchitectures. Although current silicon technology allows to build SoCs witha large number of embedded cores, the capabilities currently offered by theembedded software (e.g., in terms of OS primitives) does not allow to fullyexploit all the potential computational power; therefore, most implementationsof SoC consist of few (seldom more than 16) processor cores, for which a sharedinterconnect is perfectly suitable. The architecture of these MPSoC platforms is thus reminiscent of tradi-tional multi-processor systems, where inter-processor communication and/orsynchronization is provided through the exchange of data through shared mem-ories of different types. Generally speaking, accessing the shared memories aresignificantly slower than accesses to local ones. First, they are placed fartheraway from the processors than private memories; in fact, the latter are oftentightly coupled to the cores by means of dedicated local buses, while sharedmemories are forcedly connected to a shared bus. Moreover, accesses to theshared buses by the processors requires some form of arbitration, which mayrequire the insertion of wait cycles in case of simultaneous accesses. As aconsequence, the shared memories tend to become a major bottleneck for thebandwidth of the overall system, especially for applications in which parallelismis built around shared data. Caching of shared data might be a solution, but it raises the well-know issue ofcache coherence, i.e., the possible inconsistence between data stored in cachesof different processors. Cache coherence can be solved in hardware, yet withan extra overhead that may not be affordable in small-scale, low-cost SoC asthose considered in this work. Software-based cache coherence is also a viablesolution, but it essentially consists of limiting the caching of shared data to safetimes [2]. For applications in which parallelism is built around shared data, thisbasically amounts to avoid caching of shared data. In this paper, this will beour assumption: all accesses to shared data will always imply an access to theshared memory. Providing sufficient memory bandwidth to sustain fast program execution anddata communication/transfers is mandatory for most embedded applications.Increasing memory bandwidth can be achieved by making use of differenttypes of on-chip embedded memories, which provide shorter latencies andwider interfaces [3–5]. One typical solution used to match the computationalbandwidth with that of memory is to use multi-port memories. This solutionincreases the sustainable bandwidth by construction, since a P -port memory
  • 86allows in fact up to P accesses in parallel (i.e., in a single memory cycle).Therefore, by properly choosing the number of ports of the memory versus thenumber of processors, the issue of synchronization of simultaneous accessescan be easily solved. The adoption of multi-port memories, however, comes at the price of a sig-nificant increase in area, wiring resources, and energy consumption. On theother hand, architectures based on multi-port memories seem to be the onlyviable option in the cases where bandwidth optimization has absolute priority. In this work we propose an alternative architecture for the shared memorywhich combines the advantages, in terms of bandwidth, of the multi-port ap-proach, with the advantages, in terms of energy consumption and access time,of partitioned memories [5]. We propose the use of small, single-port mem-ory blocks as a way to achieve memory bandwidth increase together with lowenergy demand. In our scheme, the memory addressing space is mapped oversingle-port banks that can be simultaneously accessed by different processors,so as to mimic for a large fraction of the execution time the behavior of a dual-port memory. Energy efficiency is enforced by two facts: First, the single-portblocks have an energy access cost which is smaller than that of monolithic(either single or dual-port) memories; second, address mapping is application-driven, and cell access frequency data is thus used to determine the optimalsizes of the memory blocks. Based on analytical expressions for performance and energy consumptionthat allow to explore the energy-performance tradeoff, we present experimentalresults showing that the new architecture guarantees energy savings as high as69% with respect to a dual-port memory configuration (54% with respect tothe baseline, single-ported architecture), with comparable improvement of thememory bandwidth. The rest of the chapter is organized as follows. Section 5.2 provides somebackground material on memory energy modeling, multi-port memories, andapplication-driven memory partitioning. Section 5.3 describes how partitionedmemories can be used to achieve an energy-efficient shared memory architec-ture. Section 5.4 illustrates the analytical models used to drive the energy-performance exploration engine, which is discussed in Section 5.5. Section 5.6presents the optimization results for a set of standard parallel applications. Fi-nally, some concluding remarks are provided in Section BACKGROUND5.2.1 Modeling Memory Energy Unlike generic hardware modules, the energy consumption of memories isbasically independent of the input activity. What matters, in fact, is whether weare reading or writing a value from or to the memory, regardless of the value.
  • 87This property allows to model memory energy consumption in an very abstractway, by explicitly exposing two independent variables affecting it: the cost ofan access and the total number of accesses. This translates into the followingformula: ctot etot = ei (1) i=1where ctot is the total number of memory accesses, and ei is the cost of eachaccess. For the sake of simplicity, we equally weigh all accesses (i.e., we donot distinguish the cost of a read from that of a write). Equation 1 exposes the two quantities we can consider to reduce the energyconsumption of a memory system and will be used throughout the paper asa reference. Techniques for reducing memory energy can be thus classifiedaccording to which variable is optimized [6].5.2.2 Multi-Port Memories A multi-port memory is simply a memory that allows multiple simultaneousaccesses for reads and writes to any location in memory. Multi-port memo-ries are typically employed as shared memories in multiprocessor designs, andare especially popular as dual-ended FIFO buffers for bus interfacing, or forvideo/graphics buffering. Multiple simultaneous accesses are made possible by duplicating some ofthe resources required to access a cell: the address and data pins, the word-lines,and the bit-lines. Figure 5.1 shows the structure of a typical dual-port SRAMcell, and in particular the extra word-line (with the corresponding transistors)and extra bit-line.               Figure 5.1. Structure of a Dual-Port SRAM Cell. In some devices, additional overhead is also required to handle the synchro-nization of multiple writes to the same cell; this is managed through a sort ofhardware semaphore which serializes the concurrent accesses. The increase in bandwidth provided by multi-port memories comes at theprice of increased area, wiring resources and power consumption. Becauseof this considerable overhead, multi-port memories are usually limited to a
  • 88few ports (often 2, and seldom more than 4). One noticeable exception isrepresented by register files (although they are not strictly SRAMs), that aretypically highly multi-ported (even 16 or more ports) to provide very highbandwidth in superscalar processors. Multi-port memories can also be characterized by the flexibility of the ports.In some memory devices, some of the ports can be specialized, i.e., they allowonly some type of access (read or write). This fact can be expressed by writingthe number of ports P = pr + pw + prw , where the three terms denote thenumber of read, write, and read/write ports, respectively. In this work, withoutloss of generality, we will assume that pr = pw = 0, and prw = P , that is, allports can be used for any type of access at any time. When analyzing multi-port memories from the energy point of view, wemust take into account the two following non-idealities, supported by datafrom several multi-port memory providers ([7],[8],[9]). a) Energy consumption of multi-port memories does not scale linearly with the number of ports. For instance, the energy cost for accessing a dual- port memory is more than twice the energy required for accessing a single-port memory of the same size. b) When a multi-port memory is used as a shared memory in a multiproces- sor system, there are cases in which not all the ports are used simulta- neously. It may in fact happen that the access pattern of the application does not allow to group a set of accesses (from the processors) into a single, multi-port access. In these cases, we must consider the fact that energy consumption does not scale linearly with the number of ports that are accessed simultaneously. For instance, the energy cost for accessing a single port in a dual-port memory is larger than the one for accessing a single-port memory of the same size.With reference to the model of Equation 1, the use of multi-port memoriesreduces ctot , but it implies a sizable increase of the access cost ei .5.2.3 Application-Driven Memory Partitioning Partitioning a memory block into multiple blocks, based on the memory ac-cess profile, was originally proposed by Benini et al. [10]. Their techniqueexploits the fact that, due to the high locality exhibited by embedded applica-tions, the distribution of memory references is not uniform. As a consequence,some memory locations will be accessed more frequently than others. Thepartitioning is realized by splitting the address space (stored onto a single,monolithic memory block) into non-overlapping contiguous sub-spaces (storedonto several, smaller memory blocks).
  • 89 Reduction of energy consumption is achieved because of two facts. First,each block is smaller than the monolithic one, and thus it has a smaller accesscost (ei ). Second, and more relevant, only one of the blocks is active at a time.By properly partitioning the address space, it should be possible to access thesmallest blocks most of the times, and access the largest ones only occasionally. The original implementation of [10] employs a sophisticated recursive algo-rithm to determine the optimal partition with an arbitrary granularity. In thiswork, we will exploit their idea, yet without employing the same partitioningengine. As a matter of fact, in our case partitioning is driven by the accesspatterns of more than one processor. Memory partitioning specifically targets the reduction of the access cost ei ,and it does not change ctot , since it does not modify the access patterns.5.3 PARTITIONED SHARED MEMORY ARCHITECTURE The target MPSoC architecture considered is this work is depicted in Fig-ure 5.2. Each processor core has a cache and a private memory (PM) containingprivate data and code, which is accessed through a local bus. Processors arealso connected to another memory (SM), through a common global bus con-taining the data that are shared between the various threads executing on theprocessors. We do not consider here other types of interconnections, such aspoint-to-point ones (i.e., crossbars).            gure 5.2. Generic Architectural Template. In this work, starting from the assumption that the shared memory is imple-mented as a conventional on-chip, single-port memory, we aim at improvingthe performance of the accesses to the shared memory, yet in a more energy-efficient way than resorting to a multi-port memory. The proposed shared memory architecture combines the bandwidth advan-tages of multi-port memories (and thus the reduction of ctot ) with the advan-
  • 90tages, in terms of energy consumption and access time, of partitioned, single-port memories (and thus the reduction of ei ). In our scheme, the memory address space is mapped over single-port banksthat can be simultaneously accessed by the different processors, so as to mimicthe behavior of a multi-port memory for a large fraction of the execution time.Each bank covers a subset of the address space, with no replication of memorywords; therefore, the address sub-spaces are non-overlapping. The latter issueis essential to understand why the partitioned scheme can only approach theperformance of the multi-port architecture. Since the memory blocks are single-ported and contain non-overlapping subsets of addresses, simultaneous accessesfrom the processor can be parallelized only if they fit into different memoryblocks. Otherwise, the potentially parallel access must take place into twoconsecutive memory cycles. Energy efficiency is enforced by two facts: First, the single-port blocks havean energy access cost which is by far smaller than that of monolithic (eithersingle or dual-port) memories; second, address mapping is application-driven,and it accounts thus for the cell access frequency to determine the size of thememory blocks which is most suitable for memory minimization. In the following, we will restrict our analysis to systems with two proces-sors. Consequently, we will consider dual-port memories, and the partitionedarchitecture will also consists of two blocks at most. Although the concepts thatwill be discussed apply in principle to an arbitrary number of processors (withmulti-port memories and multi-bank architectures), the quantitative analysisof energy and performance strictly refers to the case of two processors (withdual-port memory and two memory blocks). A1 P1 A A1 SPM1 Port1 D P1 D1 D1 A2 DPM A2 Port2 P2 D2 P2 A SPM2 D2 D (a) (b) Figure 5.3. Dual-Port (a) and Partitioned Single-Port (b) Architectures. Figure 5.3 show a conceptual architecture of the dual-port and the partitionedsingle-port schemes. Label Ai refers to addresses from processor i, while Direfer to data to/from processor i. In the dual-port scheme (Figure 5.3-(a)), the
  • 91existence of two read/write ports allows to bind each processor to one port,realizing in fact a point-to-point interconnection. In the partitioned architecture (Figure 5.3-(b)), addresses and data must bemultiplexed (from processor to memory) or de-multiplexed (from memory toprocessor) properly, to connect the processor to the required memory block.This block diagram just shows the high-level flow of data and addresses; theactual implementation of the decoder is actually more complex, and will bediscussed in the experimental section.5.3.1 Related Work The literature on energy optimization of embedded memories is quite rich(see [6] for a comprehensive survey); however, most techniques deal with theoptimization of caches, scratch-pad memories, or off-chip memories, and multi-port memories are seldom addressed. Most energy optimizations for multi-port memories are concerned with theissue of the mapping of data structures (typically, arrays) to multi-port mem-ories, based on the access profiles of the applications. From these profiles,these techniques evaluate simultaneous array accesses (e.g., whether two ormore arrays are accessed in the same cycle), and build a so-called compatibilitygraph, which expresses the potential parallelization of accesses. The variousapproaches differ then in how this graph is used to decide the optimal allocationof array accesses to memory ports [3, 11–13]. One technique closer to the one proposed in this work has been discussedby Lewis and Brackenbury [14]. Their approach is based on the typical accesspatterns of DSP applications, and splits highly-multiported register files intomultiple banks of predefined sizes.5.4 PERFORMANCE AND ENERGY CHARACTERIZATION In this section we will derive analytical expressions for the number of memoryaccesses and for the total energy consumption for the architectures of Figure 5.3,referred to the case of a system consisting of two processors (hereafter denotedwith P1 and P2 ).5.4.1 Performance Characterization Let c1 and c2 be the number of memory accesses required by the executionof the application on processors P1 and P2 , respectively. In the following,we will use the term memory cycle instead of memory access; we adopt thisterminology in order to distinguish accesses to the shared memory that canoccur in parallel. In fact, the total number of memory accesses by a processor
  • 92is fixed (and determined by the memory access pattern of the application, whichwe do not modify); What actually changes is the time (in cycles) required toserve these accesses. Furthermore, we will denotes sets with bold symbols, andtheir cardinalities with lowercase ones. Our reference performance figure is the total number of memory cycles forthe case where shared memory is implemented as a monolithic single-portmemory. This value is cspm = c1 + c2 . Dual-Port Memory. When the shared memory is implementedby a monolithic dual-port memory, the total number of memory accesses willbe smaller than cspm because of the possibility of simultaneous accesses. Onlya fraction of the accesses, however, will occur simultaneously. As Figure 5.4 shows, this fraction can be represented in terms of set notation.We denote with Cpar the set of memory cycles that can access memory simul-taneously; Cpar consists of the union of two subsets Cpar = Cpar,1 ∪ Cpar,2 ,where Cpar,1 ⊆ C1 and Cpar,2 ⊆ C2 . These two subsets have same cardinality(i.e., cpar,1 ≡ cpar,2 ) because each element of one set matches one of the otherset to make a parallel access.      Figure 5.4. Classification of Execution Cycles. The number of cycles for the dual-port configuration is therefore: cdpm = (c1 − cpar,1 ) + (c2 − cpar,2 ) + cpar /2 (2)where cpar = cpar,1 + cpar,2 , denotes the total number of the parallel cycles.The division by two in the last term denotes the fact that parallel cycles areactually grouped in pairs, with each pair corresponding to a single memoryaccess. Equation 2 simplifies to cdpm = c1 + c2 − cpar /2, exposing the factthat the magnitude of cpar directly translates into a performance improvement. Partitioned Memory. In the case of partitioned memory, thetwo memory banks now host two non-overlapping subsets of the address space.This implies that only a subset of the cycles in Cpar can be parallelized; inparticular, accesses that fall in the same subset of addresses now need to beserialized, since the two memory blocks are single-ported.
  • 93 This further sub-setting of the cycles is depicted in Figure 5.5, using the sameset notation as above. We can notice that C1 and C2 are now both split intotwo subsets, where Ci,j denotes the cycles of processor i that fall into block j.                     Figure 5.5. Classification of Execution Cycles for the Partitioned Architecture. This induces a partition onto Cpar , as follows. The shaded areas labeledA and D in Figure 5.5 denote parallel accesses that fall into different memoryblocks: In region A (D), P1 accesses Block 1 (Block 2), and P2 accesses Block2 (Block 1). Conversely, the regions labeled B and C denote accesses thatfall in the same memory block (Block 2 for region b, and Block 1 for regionc). Cycles belonging to region B and C cause a performance penalty, because,although they can potentially occur in parallel, they must be serialized (and thusrequire two memory accesses). These subsets can be characterized by using a quantity λ, that denotes thepercentage of the cycles in Cpar that fall in distinct memory blocks (and canthus be made parallel). λ will be used in the following as a compact metric toevaluate the cost of the partition. In fact, λ depends on where how the partitionhas been made, that is, how many addresses fall in each block. Therefore, Cparconsists of λcpar cycles that can be parallelized, and (1 − λ)cpar that requirestwo separate accesses. The number of cycles of the partitioned-memory architecture cspm,part istherefore: cspm,part = (c1 − cpar,1 ) + (c2 − cpar,2 ) + λcpar /2 + (1 − λ)cpar (3) The formula simplifies to cspm,part = c1 + c2 − λcpar /2, exposing the factthat cspm,part ≥ cdpm , since λ ≤ 1. Analyzing the dependency of cspm,partversus λ, We notice that cspm,part (and thus) the performance penalty of thepartitioned scheme is minimized when λ is maximized, as expected. In par-ticular, when λ = 1, all accesses in Cpar are parallelized, and the partitionedscheme is equivalent to the dual-port memory, performance-wise. When λ = 0,all accesses by Cpar overlap on the same memory block, and the partitionedscheme is equivalent to the single-port memory architecture.
  • 945.4.2 Energy Characterization To compute energy, we stick to the high-level model of Equation 1; energyis thus simply obtained by multiplying each access for its cost. Dual-Port Memory. In this case we have to consider twotypes of access costs, depending on whether one or both ports are accessed.Total energy is obtained thus by properly weighing the terms of Equation 2: Informula: edpm = (c1 − cpar,1 ) · edpm,1 + (c2 − cpar,2 ) · edpm,1 + cpar /2 · edpm,2 (4)The term edpm,x denotes the energy per access to the memory, in which theterm x = {1, 2} in the subscript denotes the number of ports used in the access. Partitioned Memory. In the case of the partitioned memory,total energy cannot be conveniently expressed by a closed formula, for tworeasons. First, the energy per access depends on the size of the memory blockthat is accessed; the sizes of the blocks, however, are precisely the variables ofthe partitioning problem we are trying to solve. Second, we have two single-port memories, and each memory access from either processor will fall into oneof the two memory blocks. This implies that the energy per access can onlybe approximated by a “average” cost (i.e., the number of accesses to Block 1weighted by its energy cost, plus number of accesses to Block 2 weighted byits energy cost). The accurate evaluation of energy for the partitioned architecture requiresthus a simulation of the dynamic address trace of the two processors, and theapplication of Equation 1 on an access-by-access basis. Nevertheless, we can derive an approximate expression of total energy thatcan be used for a rough comparison with Equation 4: espm,part = (c1 − cpar,1 ) · espm + (c2 − cpar,2 ) · espm + (5) (1 − λ)cpar · espm + λcpar /2 · (espm1 + espm2 )The first two term (espm and espm ) are the above mentioned average accesscosts and represent the non-parallel memory accesses. espm is the cost ofaccessing either Block 1 or Block 2 (depending on the subset of addresses),when accesses are potentially parallel but must be serialized. The last termrepresents the subset of potentially parallel accesses that will access Block 1and Block 2 simultaneously (espm1 + espm2 ). Although approximate, Equation 5 allows to do some rough comparisonwith the dual-port scheme. First, all energy costs in Equation 5 are smaller thanedpm,2 , and, in most cases (when the sizes of the two blocks are of comparablesize), also smaller than edpm,1 . This implies that all four terms of Equation 5
  • 95are smaller than the corresponding ones in Equation 4, and energy is potentiallysmaller than the dual-port memory case, regardless of the value of λ. The actual dependency of espm,part on λ is not easily observable from Equa-tion 5. A large value of λ increases the probability of accessing both blocks inthe same cycle (this corresponds to the largest term (espm1 +espm2 )). Therefore,energy should be in principle reduced by choosing partitions which minimizeλ. In this case, in fact, only one of the two blocks (each one smaller than themonolithic memory) will be accessed in each cycle, thus using less energy; asmall value of λ, however, tends to increase the number of cycles, as alreadyobserved.5.5 EXPLORATION FRAMEWORK The models described in Section 5.4 show that there exists a tradeoff be-tween energy and performance in partitioning the shared memory. Althoughwe are searching for energy-efficient memory architectures, we cannot ig-nore performance implications; therefore, in order to search for the best en-ergy/performance tradeoff, we use energy/delay product (EDP) as a metric, andchoose to minimize EDP during the space exploration. Thanks to the simple models of Section 5.4, the optimization space is rela-tively small, since λ is the only parameter of the models. λ is a function of theaccess pattern of the application, but it also depends on how the address spaceis partitioned. Partitions can be characterized by the boundary address B thatsplits the address space [0, . . . , N − 1] into two sub-spaces [0, . . . , B − 1] and[B, . . . , N − 1]. Therefore, λ is also a function of B. As an example, Figure 5.6shows the behavior of λ versus B for a parallel FFT kernel; we can observe thatthe curve is not monotonic, showing the sensitivity of λ to the access pattern.                                                                                 Figure 5.6. Behavior of λ(B) vs. B. These observations leads us to the following exploration procedure, for ashared memory of N words: 1 Compute epm (λ) and cpm (λ) as in Section 5.4;
  • 96 2 For all possible values of B = 0, . . . , N − 1, Compute EDPpm (λ) as epm · cpm . EDPpm (λ(B)) is not a function, since there may be more values of B (and thus of EDP ) for a given value of λ. An example of such curve is shown in Figure 5.7, for the parallel FFT benchmark. pareto 3 Compute the function EDPpm (λ), obtained by selecting, for each value of λ, the smallest value of EDPpm (λ). EDPpm pareto contains the Pareto points of EDPpm (λ), and can possibly contain some discontinu- ities. Figure 5.8 shows the resulting curve for the FFT benchmark.               
  • Figure 5.7. Behavior of EDP (λ(B)) vs. λ.    
  •              Figure 5.8. Pareto Points of EDP (λ(B)). 4 Compute the minimum EDPmin , of this function, and let λmin the cor- responding value of λ; 5 On the λ vs. B plot, identify the corresponding value Bmin of B. In case of multiple values of B, choose the one that makes the partitions as equal (in size) as possible.
  • 975.6 EXPERIMENTAL RESULTS5.6.1 Experimental Setup We have implemented our partitioned memory scheme in ABSS [15]. ABSSis an execution-driven architectural simulator for multiprocessor systems devel-oped at Stanford University, that extends the ideas implemented in the AUG-MINT simulator. ABSS is based on the idea of augmentation, that is, theinstrumentation of the assembly code with various hooks that allow to makecontext switches to the simulator; augmentation translates the program intoa functionally equivalent program that runs on the simulated version of theprocessor. The memory architecture provided by ABSS includes both private and sharedmemory. All the memories are connected through a single shared bus. Yet,ABSS does not provide any specific predefined cache or shared bus model;rather, it a defines a specific interface to which user-defined cache and busmodels can be easily hooked. We have integrated Dinero [16] into ABSS, in order to provide accuratecache simulation data, and we have derived performance and energy modelsfor the shared memory (both single- and dual-port) by interpolation of theresults obtained from an industrial memory generator by ST Microelectronics.The target technology for all the models is 0.18µ. Concerning the benchmarks, we have used Stanford’s SPLASH suite [17]which includes a set of kernels and parallel applications widely used in theparallel computing community.5.6.2 Energy/Performance Tradeoff Analysis Table 5.1 shows energy-delay product (EDP) results for the above bench-marks, for the monolithic, single-port architecture (EDPmm ) and the parti-tioned one (EDPpm ), obtained using the exploration procedure of Section 5.5.The EDP reduction (Column ∆) ranges from 40.5% to 62.3% (50.2% on aver-age). exploration procedure also allows to compute the best performance and Theenergy points; these are summarized in Table 5.2, where performance improve-ments (number of cycles) and energy saving with respect to the monolithic,single port architecture are reported (Columns Best Performance and Best En-ergy). The comparison of Tables 5.1 and 5.2, shows that savings in the EDP ismostly due to energy savings than to performance savings. Minimum EDPpoints are in fact very close to minimum energy points, for most of the bench-marks, while performance improvements are less significant. Notice also thatonly benchmarks that exhibit a sizable amount of parallel cycles (e.g., FFT,LU-Cont, Radix) results in a sizable performance improvement. Conversely,energy does not seem to be that sensitive to the amount of parallel cycles.
  • 98Table 5.1. Energy-Delay Product Results.Application EDPmm EDPpm ∆ [%]Barnes 24987.8 11357.5 54.6FFT 6.4 3.7 41.2FMM 853.4 389.6 54.4LU 3931.3 2339.2 40.5LU-CONT 3734.4 2073.1 44.5Radix 59512.5 23180.5 61.0Volrend 869794.2 453283.8 47.9Water-N2 150460.7 56710.0 62.3Water-S 10581.2 5770.5 45.5Average 50.2Table 5.2. Optimal Performance and Energy Points.Application Best Performance [%] Best Energy [%]Barnes 1.5 54.4FFT 34.0 37.2FMM 2.1 54.4LU 10.9 40.3LU-CONT 19.8 40.5Radix 25.4 60.9Volrend 0.3 50.3Water-N2 13.9 62.3Water-S 8.7 45.9Average 13.0 49.6 Figure 5.9 shows the energy savings of the the partitioned architecture withrespect to the dual-port case. Numbers refer to best-performance points, sincewe want to reduce the performance penalty as much as possible. The savings donot include the cost of the decoding logic. The partitioned architecture resultsin an average energy saving of 56% (maximum 70%). This energy saving isachieved at an increase of the total number of memory cycles of 2.4% on average(10.1% maximum).5.6.3 Decoder Implementation The partitioned architecture requires an ad-hoc encoder which implementsthe conceptual scheme of Figure 5.3. The encoder must provide two mainfunctionalities. First, it must drive the selectors that decide to which block agiven memory access is directed; to do this, it must contain the information aboutthe boundary of the partition of the address space. Second, and more important,
  • 99          
  •           Figure 5.9. Energy Savings of the Partitioned Architecture vs. the Multi-Port One.it must handle the connection between processors and memory blocks; thisrequires a sort of arbitration mechanism that allows to serialize accesses thatare potentially parallel, but fall in the same subset of addresses (i.e., memoryblock). Figure 5.10 shows a more detailed block diagram of the encoder. It takesas inputs the addresses A1 and A2 from the two processors, the correspondingrequest signals Reqi , and the value B of the address corresponding to the parti-tion. It then generates the addresses to be sent to each memory block AB1 andAB2 , and the signals used to allow the processors to access memory Granti .The latter are both active but in the cases where potentially parallel accessesmust be serialized. The decoder contains two main blocks. The first block (RH, Request Han-dler) checks the two addresses A1 and A2 , and generates the Busyi outputs aswell as a signal that determines whether the accesses can be parallelized or not(S/N S). The other block (SEL), uses three inputs to decide to what memoryblock to send what address: the S/N S input, and the outputs A1i and A2i of twocomparators (the boxes labeled with “=”) which determine in which block A1and A2 are falling, respectively. By using the value of B as an external input, itis possible to make the decoder application-independent, and therefore to haveone single encoder for any application. We have implemented the decoder inVHDL, and synthesized it on a 0.18µm technology library by ST Microelec-tronics, using Synopsys Design Compiler. When applying the memory accesstrace of the FFT benchmark, the dissipation of the decoder is 0.35 µJ, about1.7% of total memory energy consumption (19.8 µJ).
  • 100          
  •              Figure 5.10. Block Diagram of the Decoder. Concerning delay, although the decoder is on the critical path (its delayadds up to the memory access time), this is not really an issue in the par-titioned architecture. In fact, the memory cycle time in this case is smallerthan that of the dual-port case, since we are accessing smaller memory blocks.Quantitatively, the partitioned architecture results in a slack equal to ddpm −max( dspm,1 , dspm,2 ), where di denotes access time to the corresponding mem-ory block. The delay of the decoder obtained from synthesis is 310ps, wellwithin this slack.5.7 CONCLUSIONS We have proposed an energy-efficient alternative to multi-port memories suit-able for the implementation of the shared memory of multi-processor systems-on-chip. The architecture is based on application-driven partitioning of theaddress space into multiple banks. The target of the architecture is to achieve little or no performance penaltywith respect to multi-port memories; therefore, we pursue maximum perfor-mance partitioning solutions, corresponding the case where the chance of par-allelizing the accesses is maximized. The architecture can be enhanced so thatzero performance penalty is achieved, thank to the use of an extra memorybuffer. Experiments on a set of parallel benchmarks has shown average energy-delayproduct (EDP) reductions of 50% on average, with respect to the baseline caseof a single-port memory, and energy savings of 56%, with respect to the caseof a multi-port memory, with an average 2% performance penalty.
  • 101References[1] L. Benini, G. De Micheli, “Networks on Chips: A New SoC Paradigm,” IEEE Computer, Vol. 35, No. 1, pp. 70–78, January 2002.[2] P. Stenstr¨ m, “A Survey of Cache Coherence Schemes for Multiprocessors,” o IEEE Computer, Vol. 23, No. 6, June 1990, pp. 12–24.[3] F. Catthoor, et al. Custom Memory Management Methodology Explo- ration for Memory Optimization for Embedded Multimedia System Design, Kluwer Academic Publishers, 1998.[4] P. Panda, N. Dutt, Memory Issues in Embedded Systems-on-Chip Optimiza- tion and Exploration, Kluwer Academic Publishers, 1999.[5] A. Macii, L. Benini, M. Poncino, Memory Design Techniques for Low- Energy Embedded Systems, Kluwer Academic Publishers, 2002.[6] L. Benini, A. Macii, M. Poncino, “Energy-Aware Design of Embedded Memories: A Survey of Technologies, Architectures and Optimization Techniques”, ACM Transactions on Embedded Computing Systems, Vol. 2, No. 1, Feb. 2003, pp. 5–32.[7] Cypress Semiconductor, http://www.cypress.com/products.[8] Integrated Devices Technology, http://www.idt.com/products/ multi port.html.[9] Artisan Components, http://www.artisan.com/products/ memory.html.[10] L. Macchiarulo, A. Macii, L. Benini, M. Poncino, “Layout-Driven Mem- ory Synthesis for Embedded Systems-on-Chip," IEEE Transactions on Very Large Scale Integration (VLSI), Vol. 10, No. 2, pp. 96-105, April 2000[11] P.R. Panda, N.D. Dutt, “Behavioral Array Mapping into Multiport Mem- ories Targeting Low-Power,” VLSI’97: International Conference on VLSI Design, Jan. 1997, pp. 268–272.[12] P.R. Panda, L. Chitturi, “An Energy-Conscious Algorithm for Memory Port Allocation,” ICCAD’02: International Conference on Computer Aided Design, Nov. 2002, pp. 572–576.[13] W.-T. Shiue, C. Chakrabarti, “Low-Power Multi-Module, Multi-Port Memory Design for Embedded Systems,” Journal of VLSI Signal Process- ing, pp.167-178, Nov 2001.[14] M. Lewis, L. Brackenbury, “Exploiting Typical DSP Data Access Patterns and Asynchrony for a Low-Power Multi-ported Register Bank,” ASYNC’01: International Symposium on Asynchronous Circuits and Systems, March 2001, pp. 4–14.[15] D. Sunada, D. Glasco, M. Flynn, ABSS v2.0: A SPARC Simulator, Tech- nical Report CSL-TR-98-755, CSL, Stanford University, April 1998.
  • 102[16] M. D. Hill, J. Elder, DineroIV Trace-Driven Uniprocessor Cache Simula- tor, www.cs.wisc.edu/markhill/DineroIV, 1998.[17] J. P. Singh, W.-D. Weber, A. Gupta, “SPLASH: Stanford Parallel Appli- cations for Shared-Memory”, Computer Architecture News, Vol. 20, No. 1, pages 5-44, March 1992.
  • 103Chapter 6TUNING CACHES TO APPLICATIONS FORLOW-ENERGY EMBEDDED SYSTEMSAnn Gordon-Ross1, Chuanjun Zhang1, Frank Vahid1,2, and Nikil Dutt21 University of California, Riverside;2 University of California, IrvineAbstract The power consumed by the memory hierarchy of a microprocessor can contribute to as much as 50% of the total microprocessor system power, and is thus a good candidate for power and energy optimizations. We discuss four methods for tuning a microprocessors’ cache subsystem to the needs of any executing application for low-energy embedded systems. We introduce on- chip hardware implementing an efficient cache tuning heuristic that can automatically, transparently, and dynamically tune a configurable level-one cache’s total size, associativity and line size to an executing application. We extend the single-level cache tuning heuristic for a two-level cache using a methodology applicable to both a simulation-based exploration environment and a hardware-based system prototyping environment. We show that a victim buffer can be very effective as a configurable parameter in a memory hierarchy. We reduce static energy dissipation of on-chip data cache by compressing the frequent values that widely exist in a data cache memory.Keywords: Cache; configurable; architecture tuning; low power; low energy; embedded systems; on-chip CAD; dynamic optimization; cache hierarchy; cache exploration; cache optimization; victim buffer; frequent value.6.1 INTRODUCTION The power consumed by the memory hierarchy of a microprocessor cancontribute to 50% or more of total microprocessor system power [1]. Such alarge contributor to power is a good candidate for power and energyoptimization. The design of the caches in a memory hierarchy plays a majorrole in the memory hierarchy’s power and performance. Tuning cache design parameters to the needs of a particular applicationor program region can save energy. Cache design parameters include: cachesize, meaning the total number of data byte storage; cache associativity,meaning the number of tag and data ways simultaneously read per cache
  • 104 access; cache line size, meaning the number of bytes in a block when moving data between cache and the next memory level; and victim buffer use, meaning a small fully-associative buffer storing recently-evicted cache data lines. Every application has different cache requirements that cannot be efficiently satisfied with one predetermined cache configuration. For instance, different applications have vastly different spatial and temporal locality and thus have different requirements [2] with respect to cache size, cache line size, cache associativity, victim buffer configuration, etc. In addition to tunable cache parameters, widely existing frequent values in data caches for some applications can enable data encoding within the cache for reduced power consumption. We define cache tuning as the task of choosing the best configuration of cache design parameters for a particular application, or for a particular phase of an application, such that performance, power and/or energy are optimized. New technologies enable cache tuning. Core-based processors allow a designer to choose a particular cache configuration [3-7]. Some processor designs allow caches to be configured during system reset or even during runtime [2,8,9]. Manual tuning of the cache is hard. A single-level cache may have many tens of different cache configurations, and interdependent multi-level caches may have thousands of cache configurations. The configuration space gets even larger if other dependent configurable architecture parameters are considered, such as bus and processor parameters. Exhaustively searching the space may be too slow even if fully automated. With possible average energy savings of over 40% through tuning [2,10], we sought to develop automated cache tuning methods. In this chapter, we discuss four methods of cache tuning for energy savings. We discuss an in-system method for automatically, transparently, and dynamically tuning a level-one cache; an automatic tuning methodology for two-level caches applicable to both a simulation-based exploration environment or a hardware-based prototyping environment; a configurable victim buffer; and a data cache that encodes frequent data values. 6.2 BACKGROUND – TUNABLE CACHE PARAMETERS Many methods exist for configuring a single level of cache to a particular application during design time and in-system during runtime. Cache configuration can be specified during design time for many commercial soft cores from MIPS [6], ARM [5], and Arc [4] and for environments such as Tensilica’s Xtensa processor generator [7] and Altera’s Nios embedded processor system [3].
  • 105 Configurable cache hardware also exists to assist in cache configuration.Motorola’s M*CORE [9] processors offer way configuration which allowsthe ways of a unified data/instruction cache to individually be specified aseither data or instruction ways. Additionally, ways may be shut downentirely. Way shut-down is further explored by Albonesi [8] to reducedynamic power by an average of 40%. An adaptive cache line sizemethodology is proposed by Veidenbaum et al.[11] to reduce memory trafficby more than 50%. Exhaustive search methods may be used to find optimal cacheconfigurations, but the time required for an exhaustive search is oftenprohibitive. Several tools do exist for assisting designers in tuning a singlelevel of cache. Platune [12] is a framework for tuning configurable system-on-a-chip (SOC) platforms. Platune offers many configurable parametersbeyond just cache parameters, and prunes the search space by isolatinginterdependent parameters from independent parameters. The level onecache parameters, being dependent, are explored exhaustively. Heuristic methods exist to prune the search space of the configurablecache. Palesi et al. [13] improves upon the exhaustive search used in Platuneby using a genetic algorithm to produce comparable results in less time.Zhang et al. [14] presents a cache configuration exploration methodologywherein a cache exploration component searches configurations in order oftheir impact on energy, and produces a list of Pareto-optimal pointsrepresenting reasonable tradeoffs in energy and performance. Ghosh etal.[15] uses an analytical model to efficiently explore cache size andassociativity and directly computes a cache configuration to meet thedesigners’ performance constraints. Few methods exist for tuning multiple levels of a cache hierarchy.Balasubramonian et al. [10] proposes a hardware-based cache configurationmanagement algorithm to improve memory hierarchy performance whileconsidering energy consumption. An average reduction in memory hierarchyenergy of 43% can be achieved with a configurable level two and level threecache hierarchy coupled with a conventional level one cache.6.3 A SELF-TUNING LEVEL ONE CACHE ARCHITECTURE Tuning a cache to a particular application can be a cumbersome task leftfor designers even with the advent of recent computer-aided design (CAD)tuning aids. Large configuration spaces may take a designer weeks ormonths to explore and with a small time-to-market, lengthy tuning iterationsmay not be feasible. We propose to move the CAD environment on-chip,eliminating designer effort for cache tuning. We introduce on-chip hardware
  • 106 implementing an efficient heuristic that automatically, transparently, and dynamically tunes the cache to the executing program to reduce energy [16]. 6.3.1 Configurable Cache Architecture The on-chip hardware tunes four cache parameters in the level-one cache: cache line size (64, 32, or 16 bytes), cache size (8, 4, or 2 Kbytes), associativity (4, 2, or 1-way), and cache way prediction (on or off). Way prediction is a method for reducing set-associative cache energy, in which one way is initially accessed, and other ways accessed only upon a miss. I$ Micro- Off chip Tuner Memory processor D$ Figure 6-1. Self-tuning cache architecture The exploration space is quite large, necessitating an efficient exploration heuristic implemented with specialized tuning hardware, as illustrated in Figure 6-1. The tuning phase may be activated during a special software- selected tuning mode, during startup of a task, whenever a program phase change is detected, or at fixed time intervals. The choice of approach is orthogonal to the design of the self-tuning architecture itself. The cache architecture supports a certain range of configurations [2]. The base level-one cache of 8 Kbytes consists of four banks that can operate as four ways. A special configuration register allows the ways to be concatenated to form either a direct-mapped or 2-way set associative 8 Kbyte cache. The configuration register may also be configured to shut down ways, resulting in a 4 Kbyte direct-mapped or 2-way set associative cache or a 2 Kbyte direct-mapped cache. Specifically, due to the bank layout for way shut down, 2 Kbyte 2- or 4-way set associative and 4 Kbyte 4-way set associative caches are not possible using the configurable cache hardware. 6.3.2 Heuristic Development Through Analysis A naïve tuning approach would simply try all possible combinations of configurable parameters in an arbitrary order. For each configuration, the miss rate can be measured and used to estimate the energy consumption of the particular cache configuration. After all configurations are executed, the approach would simply choose the configuration with the lowest energy
  • 107consumption. However, such an exhaustive method may involve theinspection of too many configurations. Therefore, we wish to develop acache tuning heuristic that minimizes the number of configurations explored. When developing a good heuristic, the parameter (cache size, line size,associativity, or way prediction) with the largest impact in performance andenergy would likely be the best parameter to search first. We analyzed eachparameter to determine the parameter’s impact on miss rate and energy byfixing three parameters and varying the third. We observed that varying the cache size had the largest average impacton energy and miss rate – changing the cache size can impact the energy bya factor of two or more. From our analysis, we developed a search heuristicthat first determines the best cache size, determines the best line size, thenthe best associativity, and finally, if the best associativity is greater than one,our heuristic determines whether to use way prediction or not.6.3.3 Search Heuristic The heuristic developed based on the importance of parameters issummarized below:1. Begin with a 2 Kbyte, direct-mapped cache with a 16 byte line size. Increase the cache size to 4 Kbytes. If the increase in cache size causes a decrease in energy consumption, increase the cache size to 8 Kbytes. Choose the cache size with the best energy consumption.2. For the best cache size determined in step 1, increase the line size from 16 bytes to 32 bytes. If the increase in line size causes a decrease in energy consumption, increase the line size to 64 bytes. Choose the line size with the best energy consumption.3. For the best cache size determined in step 1 and the best line size determined in step 2, increase the associativity to 2 ways. If the increase in associativity causes a decrease in energy consumption, increase the associativity to 4 ways. Choose the associativity with the best energy consumption.4. If step (3) determined the best associativity to be greater than 1, determine if enabling way prediction results in energy savings. The cache tuning heuristic can be implemented in either software orhardware. In a software-based approach, the system processor would executethe search heuristic. Executing the heuristic on the system processor wouldnot only change the runtime behavior of the application but also affect thecache behavior, possibly resulting in the search heuristic choosing a non-
  • 108 optimal cache configuration. Therefore, we prefer a hardware-based approach that does not significantly impact overall area or power. 6.3.4 Experiments and Results We simulated numerous Powerstone [9] and MediaBench [18] benchmarks using SimpleScalar [19], a cycle-accurate simulator that includes a MIPS-like microprocessor model, to obtain the number of cache accesses and cache misses for each benchmark and configuration explored. For power dissipation, we considered both static power dissipation due to leakage current and dynamic power dissipation due to logic switching current and the charging and discharging of the load capacitance. We obtain the energy of a cache hit from our own CMOS 0.18 µm layout of our configurable cache (we found our energy values correspond closely with CACTI values). We obtain the off-chip memory access energy from a standard Samsung memory, and the stall energy from a 0.18 µm MIPS microprocessor. Furthermore, we obtained the power consumed by our cache tuner, through simulation of a synthesized version of our cache tuner written in VHDL. Table 6-1. Results of search heuristic. Ben. is the benchmark considered, cfg. is the cache configuration selected, No. is the number of configurations examined by our heuristic, and E% is the energy savings of both the I-cache and D-cache. Ben. I-cache cfg No. D-cache cfg No. I-cache E% D-cache E% padpcm 8K_1W_64B 7 8K_1W_32B 7 23% 77% crc 2K_1W_32B 4 4K_1W_64B 6 70% 30% auto 8K_2W_16B 7 4K_1W_32B 6 3% 97% bcnt 2K_1W_32B 4 2K_1W_64B 4 70% 30% bilv 4K_1W_64B 6 2K_1W_64B 4 64% 36% binary 2K_1W_32B 4 2K_1W_64B 4 54% 46% blit 2K_1W_32B 4 8K_2W_32B 8 60% 40% brev 4K_1W_32B 6 2K_1W_64B 4 63% 37% g3fax 4K_1W_32B 6 4K_1W_16B 5 60% 40% fir 4K_1W_32B 6 2K_1W_64B 4 29% 71% jpeg 8K_4W_32B 8 4K_2W_32B 7 6% 94% pjpeg 4K_1W_32B 6 4K_1W_16B 5 51% 49% optimal 4K_2W_64B ucbqsort 4K_1W_16B 6 4K_1W_64B 6 63% 37% tv 8K_1W_16B 7 8K_2W_16B 7 37% 63% adpcm 2K_1W_16B 5 4K_1W_16B 5 64% 36% epic 2K_1W_64B 5 8K_1W_16B 6 39% 61% g721 8K_4W_16B 8 2K_1W_16B 3 15% 85% pegwit 4K_1W_16B 5 4K_1W_16B 5 37% 63% mpeg2 4K_1W_32B 6 4K_2W_16B 6 40% 60% optimal 8K_2W_16B Average 5.8 Average: 5.4 45% 55%
  • 109 Table 6-1 shows the results of our search heuristic, for instruction anddata cache configurations. Our search heuristic is quite effective: it searcheson average only 5.8 configurations, compared to 27 configurations for anexhaustive approach. Furthermore, our heuristic finds the optimalconfiguration in nearly all cases. For the two data cache configurationswhere the heuristic does not find the optimal, pjpeg and mpeg2, theconfiguration found is only 5% and 12% worse than the optimal,respectively. On average, the dynamic self-tuning cache can reduce memory-access energy by 45% to 55%. Additionally, be observed that way predictionis only beneficial for instruction caches and that only a 4-way set associativeinstruction cache has lower energy consumption when way prediction isused. However, for the benchmarks we examined, the cache configurationswith the lowest energy dissipation were mostly direct mapped caches whereway prediction is not applicable. To determine the area and power overhead of our cache tuner, wedesigned the cache tuner hardware using VHDL and synthesized the tunerusing Synopsys Design Compiler. The total tuner size was about 4,000 gates,or 0.039 mm2 in 0.18 µm CMOS technology. Compared to the reported sizeof the MIPS 4Kp with caches [20], this represents an increase in area of justover 3%. The power consumption of the cache tuner is 2.69 mW at 200MHz, which is only 0.5% of the power consumed by a MIPS processor.Furthermore, we only use the tuning hardware during the tuning stage; thetuner can be shutdown after the best configuration is determined, therebyminimizing the effects of additional static power dissipation due to the tuner.6.4 AUTOMATIC TUNING OF A TWO-LEVEL CACHE ARCHITECTURE – THE TCAT In the previous section, we described an automatic method for tuning asingle level of cache in system during run-time. We extend the single levelcache tuner to tune two-level caches to embedded applications for reducedenergy consumption [21]. This method is applicable to both a simulation-based exploration environment and a hardware-based prototypingenvironment. We present the two-level cache tuner, or TCaT – a heuristic forsearching the huge solution space of possible configurations. The heuristicinterlaces the exploration of the two cache levels and searches the variouscache parameters in a specific order based on their impact on energy.
  • 110 6.4.1 Configurable Cache Architecture The configurable caches in each of the two cache levels explored here are based on the configurable cache architecture described for a single level configurable cache in Section 6.3.1. The target architecture for our two-level cache tuning heuristic contains separate level one instruction and data caches and separate level two instruction and data caches. For the first level cache, we explore the same search space as the single level cache tuner: cache line size (64, 32, or 16 bytes), cache size (8, 4, or 2 Kbytes), and associativity (4, 2, or 1-way). For the second level of cache, we expand the cache size to a possible 64, 32, or 16 Kbytes while the line size and associativity parameters are the same. We do not explore way prediction with the TCaT. An exhaustive exploration of all cache configurations for a two level cache hierarchy is too costly. For a single level separate instruction and data cache design, an exhaustive exploration would explore a total of 28 different cache configurations. However, the addition of a second level of hierarchy raises the number of cache configurations to 432. Nevertheless, for comparison purposes, we determined the optimal cache configuration for each benchmark by generating exhaustive data. It took over one month of continual simulation time on an UltraSparc compute server to generate the data for our nine benchmarks. In addition, we have chosen a base cache hierarchy configuration consisting of an 8 Kbyte, 4-way set associative level-one cache with a 32 byte line size, and a 64 Kbyte 4-way set associative level two cache with a 64 byte line size – a reasonably common configuration. 6.4.2 Initial Two-Level Cache Tuning Heuristic – Search Each Level Independently Initially, we extended the heuristic described in Section 6.3.3 for a two- level cache by tuning the level-one cache while holding the level-two cache at the smallest size, then tuning the level-two cache using the same heuristic. We applied the initial heuristic to the benchmarks and found that this heuristic did not perform well for two levels (the original heuristic was intended for only one level, where it works well). The cache configuration determined by our initial heuristic consumed, on average over all benchmarks, 1.41 times more energy than the optimal configuration. In the worst case, our initial heuristic found a cache configuration using 2.7 times more energy than the optimal configuration. In one benchmark, the initial heuristic found a cache configuration that was worse than the base cache. The naïve assumption that the two levels of cache could be configured independently was the reason that our initial heuristic did not perform well
  • 111for a two level system. In a two-level cache hierarchy, the behavior of eachcache level directly affects the behavior of the other level. For example, themiss rate of the level one cache does not solely determine the performance ofthe level two cache. The performance of the level two cache is alsodetermined by what values are missing in the level one cache. To fullyexplore the dependencies between the two levels, we decided to explore bothlevels simultaneously.6.4.3 The Two-Level Cache Tuner - TCaT To more fully explore the dependencies between the two cache levels, weexpanded our initial heuristic to interlace the exploration of the level one andlevel two caches. Instead of entirely configuring the level one cache beforeconfiguring the level two cache, the interlaced heuristic explores oneparameter for both levels of cache before exploring the next parameter,while adhering to the parameter ordering of the initial heuristic. The basicintuition behind our heuristic is that interlacing the exploration allows forbetter modeling and tuning of the interdependencies between the differentlevels of cache hierarchy. We applied the interlaced heuristic to thebenchmarks and found that the interlaced heuristic performed much betterthan the initial heuristic, but there was still much room for improvement. We examined the cases where the interlaced heuristic did not yield theoptimal solution. We discovered that in these cases, the optimal was notbeing reached for two reasons. First, the initial heuristic did not fully exploreeach parameter. For instance, if an increase from a 2 Kbyte to 4 Kbyte cachesize did not yield an improvement in energy, an 8 Kbyte cache size was notexamined. The second reason the optimal configuration was not being foundwas not due to a failure in the heuristic, but rather due to the limitations seton certain cache configurations by the configurable cache itself. Forexample, in the level two cache, if a 16 Kbyte cache is chosen as the bestsize, the only associativity available is a direct-mapped cache. With noenergy improvement by increasing the cache from a 16 Kbyte direct-mappedto a 32 Kbyte direct-mapped cache, no other associativities are searched bythe previous heuristics. To allow for all associativities to be searched, weadded a final adjustment to the associativity search step of the interlacedheuristic with full parameter exploration. The final adjustment allows thecache size to be increased for both the level one and level two caches inorder to search larger associativities. We refer to this final heuristic as thetwo-level cache tuner - the TCaT.
  • 112 6.4.4 Experiments and Results The experimental setup and energy calculations are the same as those described in Section 6.3.4. We explored nine different benchmarks obtained from MediaBench [18] and EEMBC [22] benchmarks suites. 1.2 1 Base Cache 0.8 0.6 Initial Heuristic 0.4 TCaT 0.2 0 Optimal AIFFTR01 AIFIRF01 BITMNP01 TTSPRK01 average g721 pegwit IDCTRN01 PNTRCH01 rawcaudio Figure 6-2. Energy consumption for the initial heuristic cache configuration, the TCaT cache configuration, and the optimal cache configuration, normalized to the base cache configuration for each benchmark. Figure 6-2 shows the results for the initial heuristic and the TCaT for each benchmark. The energy consumptions have been normalized to the base cache configuration for each benchmark’s cache hierarchy. The results show that the TCaT finds the optimal cache configuration in most cases. Compared to the base cache configuration and averaged over all benchmarks, the initial heuristic achieves an average energy savings of 32% while the TCaT achieves an average energy savings of 53%. Additionally, we found that for every benchmark, there is no loss of performance due to cache configuration for optimal energy consumption. In fact, the benchmarks receive an average of a 28% speedup, which we found was due to the tuning of the cache line size. Furthermore, the TCaT reduces the configuration search space significantly. The exhaustive approach for separate instruction and data caches for a two level cache hierarchy explores 432 cache configurations. The improved heuristic explores only 28 cache configurations, or only 6.5% of the search space. This reduction in the search space speeds up both a simulation approach and a hardware-based prototyping platform approach.
  • 1136.5 USING A VICTIM BUFFER IN AN APPLICATION SPECIFIC MEMORY HEIRARCHY In addition to tuning cache parameters such as cache size, line size, andassociativity, the cache subsystem can include a configurable victim bufferwhich can be beneficial in systems with a direct-mapped cache. Direct-mapped caches are popular in embedded microprocessor architecture due totheir simplicity and good hit rates for many applications. A victim buffer is asmall fully-associative cache, whose size is typically 4 to 16 cache lines,residing between a direct-mapped L1 cache and the next level of memory.The victim buffer holds lines discarded after an L1 cache miss. The victimbuffer is checked whenever there is an L1 cache miss, before going to thenext level memory. If the desired data is found in the victim buffer, the datain the victim buffer is swapped back to the L1 cache. Jouppi [23] reportedthat a four-entry victim buffer could reduce 20% to 95% of the conflictmisses in a 4 Kbyte direct-mapped data cache. Albera and Bahar [24]evaluated the power and performance advantages of a victim buffer in a highperformance superscalar, speculative, out-of-order processor. They showedthat adding a victim buffer to an 8 Kbyte direct-mapped data cache results in10% energy savings and 3.5% performance improvements on average for theSpec95 benchmark suite. A victim buffer improves the performance and energy of a direct-mappedcache on average, but for some applications, a victim buffer actuallydegrades performance without much or any energy savings, as we will showlater. Such degradation occurs when the victim buffer hit rate is low.Checking a victim buffer requires an extra cycle after an L1 miss. If thevictim buffer hit rate is high, that extra cycle actually prevents dozens ofcycles for accessing the next level memory. But if the buffer hit rate is low,that extra cycle does not save much and thus is wasteful. Whether a victimbuffer’s hit rate is high or low is dependent on what application is running.Such performance overhead may be one reason that victim buffers are notalways included in embedded processor cache architectures. In this section, we will show that treating the victim buffer as aconfigurable memory parameter to a direct-mapped cache is superior toeither using a direct-mapped cache without a victim buffer or using a direct-mapped cache with an always-on victim buffer [25]. Furthermore, we showthat a victim buffer parameter is even useful with a cache that itself is highlyparameterized.
  • 114 6.5.1 Victim Buffer as a Cache Parameter We consider adding a victim buffer to both core-based and pre-fabricated platform based design situations. A core-based approach involves incorporating a processor (core) into a chip before the chip has been fabricated, either using a synthesizable core (soft core) or a layout (hard core). In either case, most core vendors allow a designer to configure the level 1 cache’s total size (typical sizes range from no cache to 64 Kbyte), associativity (ranging from direct mapped to 4 or 8 ways), and sometimes line size (ranging from 16 bytes to 64 bytes). Other parameters include use of write through, write back, and write allocate policies for writing to a cache, as well as the size of a write buffer. Adding a victim buffer to a core-based approach is straightforward, involving simply including or not including a buffer into the design. A pre-fabricated platform is a chip that has already been designed, but is intended for use in a variety of possible applications. To perform efficiently for the largest variety of applications, recent platforms come with parameterized architectures that a designer can configure for his/her particular set of applications. Recent architectures include cache parameters [2,8,9] that can be configured by setting a few configuration register bits. We therefore developed a configurable victim buffer that could be turned on or off by setting bits in a configuration register. 6.5.2 Experiments and Results The experimental setup and energy calculations are the same as those described in Section 6.3.4. The benchmarks examined include programs from the Powerstone [9], MediaBench [18], and Spec2000 [26] benchmark suites. Victim Buffer with a Direct-Mapped Cache Figure 6-3 shows the performance and energy improvements when adding an always-on victim buffer to a direct-mapped cache. Performance is the program execution time. Energy is estimated as described in section 6.3.4. 0% represents the performance and energy consumption of an 8 Kbyte direct-mapped cache. From Figure 6-3, we see that a victim buffer improves both performance and energy for some benchmarks, like mpeg, epic, and adpcm. For other benchmarks, energy is not improved but performance is degraded, as for vpr, fir, and padpcm. A victim buffer should be excluded or turned off for these benchmarks. Some benchmarks, like jpeg, parser, and auto2, yield some energy savings at the expense of some performance degradation using a victim buffer – a designer might choose whether to
  • 115include/exclude or turn on/off the buffer in these cases depending onwhether energy or performance is more important. 16% 21% 24% 38% 43% 60% 12% performance 8% energy 4% 0% -4% bilv blit binary g721 pegwit cbqsort pjeg jpeg padpcm v42 auto2 mpeg g3fax fir adpcm epic vpr bcnt brev mcf art parser crcFigure 6-3. Performance and energy improvements when adding a victim buffer to an 8 Kbytedirect-mapped cache. Positive values mean the victim buffer improved performance orenergy, with 0% representing an 8 Kbyte direct-mapped cache without a victim buffer.Benchmarks with both bars positive should turn on the victim buffer, while those withnegative performance improvement and little or no energy improvement should turn off thevictim buffer. Victim Buffer with a Parameterized Cache Figure 6-4 shows the performance and energy improvement of adding avictim buffer to a parameterized cache having the same configurabilitydescribed by Zhang et. Al. [2] 0% represents the performance and energy ofthe original configurable cache when tuned optimally to a particularapplication. The bars represent the performance and energy of theconfigurable cache when optimally tuned to an application assuming avictim buffer exists and is always on. The optimal cache configurations for agiven benchmark are usually different for each of the two cases (no victimbuffer versus always-on victim buffer). We see that, even though the configurable cache already representssignificant energy savings compared to either a 4-way or direct-mappedcache [2], a victim buffer extends the savings of a configurable cache by alarge amount for many examples. For example, a victim buffer yields anadditional 32%, 43%, and 23% energy savings for benchmarks adpcm, epic,and mpeg2. The savings of adpcm and epic come primarily from the victimbuffer that reduces the visits to off-chip memory. The saving of epic comesprimarily from the victim buffer enabling us to configure the configurablecache to use less associativity without increasing accesses to the nextmemory level. Yet, for other benchmarks, like adpcm, auto2 and vpr, the
  • 116 victim buffer yields performance overhead with no energy savings and thus should be turned off. 12% 32% 43% 23% performance energy 8% 4% 0% -4% bilv pegwit pjeg jpeg blit epic g3fax mcf padpcm vpr v42 auto2 Figure 6-4. Performance and energy improvements when adding a victim buffer to an 8 Kbyte configurable cache. 0% represents a configurable cache without a victim buffer, tuned optimally to the particular benchmark. 6.6 LOW STATIC-POWER FREQUENT-VALUE DATA CACHES Recently, a frequent value (FV) low power data cache design was proposed based on the observation that a major portion of data cache accesses involves frequent values, which can be dynamically captured [27]. Frequent values are encoded in the cache, occupying only a few bits. We improve upon previous FV data caches by reducing static power by shutting off the unused bits in the larger sub-array for encoded frequent values [28]. Since frequent values are stored in encoded form using only the few bits in the smaller sub-array, the remaining bits in the larger sub-array serve no purpose as long as the value stays frequent. Such shutoff may be beneficial since FVs occupy many words in data caches [27]. Furthermore, the original FV low power cache design suffers from an extra cycle when reading non-FVs [27], which account for 68% of all data cache accesses, resulting in a 5% increase in execution time. We used circuit design to remove the extra cycle. 6.6.1 Overview of Original FV Cache Design In this section, we give a brief overview of the original FV data cache designed by Yang and Gupta [27]. The FV cache was proposed based on the observation that a small number of distinct frequently occurring data values often occupy a large portion of program memory data spaces and therefore account for a large portion of memory accesses [27]. This frequent value phenomenon was
  • 117exploited in designing a data cache that trades off performance with energyefficiency. From the perspective of the frequent value cache, data values are dividedinto two categories: a small number of frequent values, in our case 32 FVs,and all remaining values that are referred to as non-frequent values. Thefrequent values are stored in encoded form and therefore can be representedin 5 bits; the non-frequent values are stored in unencoded form in 32 bitwords. Additionally, a flag bit is needed for each word in the cache todetermine if the value stored in that location is encoded or not. The set offrequent values remains fixed for a given program run. When reading a word from the cache, initially we simply read from thelow-bit array. Since every word read out contains a flag bit, the flag isexamined to determine what comes next. The flag being 1 means the desiredword is in un-encoded form, so the remaining bits should be read out fromthe high-bit array to form the original value. On the other hand, the flagbeing 0 means that the desired word is a frequent value and stored inencoded form. In this case, the access proceeds to decode the value. Sincethe access to the high-bit array is avoided, cache activity is reduced. A write to the FV cache is performed as follows. Before a value iswritten, it is first encoded through an encoder. If encoding is successful, itmeans that the value is a frequent value and thus a 5-bit code is stored in thelow-bit array and the flag bit is cleared. In this case, accessing the high-bitarray is avoided. If the encoding fails, the value to be written is a non-frequent value and thus both low-bit and high-bit data arrays are accessed aswell as the flag bit being set. Note that writing non-FVs does not need totake two cycles as does reading non-FVs, because the value is encoded earlyin the pipeline and thus the decision of driving one array or two is clearbefore the access.6.6.2 Improving the FV Cache Design The FVs are not only accessed frequently, but also distributed widely incaches [29]. This phenomenon provides a good opportunity for reducingstatic power. Our approach is the following. Since the 32-bit FVs areencoded in 5 bits, the remaining 27 bits do not store any useful information.Therefore, they can be shut down to save static power and as long as a valuestays frequent, static power is saved. The overall savings depend on theoccupancy of FVs in the cache. Our studies show that on average nearly halfof the cache content contains FVs, which indicates the benefit of reducingstatic power through finding FVs. The flag bits are initially set to 1, which means initially all words arenon-FVs. Any data to the data cache is checked with the FV encoder. If the
  • 118 word is an FV, the corresponding flag bit is set to 0 and this cache word is encoded and stored in the 5-bit array. At the same time, the flag bit turns off the 27-bit portion of the word. Similarly, on reading FVs, only the 5-bit portion is read and the 27-bit portion is gated off using the flag bit. On a non-FV read or write, the flag bit is set to 0 and the original 32 bits are written into the cache as usual. Our new circuit design improves the original FV cache design in that there is no extra delay in determining accesses of the 27-bit portion. 6.6.3 Designers’ Choices of Using the FV Cache We have described a low static power FV cache. When utilized into a processor system, the FV cache can be designed with different degrees of complexity and flexibility. In this section, we provide three approaches that are suitable for a variety of processors targeting different types of applications. Essentially, the complexity comes from how FVs are identified and if they are allowed to vary for different applications. As always, the more flexibility the processor provides, the more complex the FV cache is. The first approach is appropriate to application specific processors. Since only a single type of application runs on the processor, its FVs tend to be stable over time. In such cases, the FVs can be first obtained from a profiling run through simulations, and then synthesized into the cache as part of the cache data storage. The advantage of this approach is that once the FVs are hard coded on-chip, the cache does not perform operations other than reads. Thus, the logic of this component is simple and can be designed to consume minimum power. The second approach extends the first one with the ability of changing the FVs according to different applications. This approach is suitable for a multi-task environment in which the processor runs multiple programs instead of single program. Each program’s FVs are still obtained off-line. Instead of synthesizing the FVs on-chip, a register file may be used to store FVs so that they can be rewritten on each activation of a different program. The size of the register file depends on the number of FVs of interest to the designer, which is heavily dependent on each program’s behavior. The third approach provides the maximum flexibility in maintaining FVs. According to a previous study [29], some programs’ FVs are sensitive to different inputs. This suggests that another dimension of varying FVs might be added into the design. Since it is infeasible to profile every program on all possible inputs to catch FVs, detecting FVs on-line would be useful. Thus, on top of the second approach, the register file could be extended to dynamically capture FVs using extra logic. In the scheme proposed by Yang and Gupta [27], an inexpensive hardware FV finder was developed that
  • 119monitored cache accesses. The FV finder was turned on for only the first 5%of memory accesses assuming that the total memory access numbers areknown a priori. After that, the FVs were captured in the finder andtransmitted to the cache so that the cache starts operating as an FV cache.The energy overhead of the finder was estimated to be 0.3%-6.1% of the L1D-cache (8 Kbyte to 64 Kbyte caches were tested). The area overhead issimilar to our second approach, and thus modest. One potential issue is thatthe FV finder described detects frequently accessed values, which may ormay not correspond to frequently distributed values in memory, though theyusually are the same. We leave an FV finder for frequently distributed valuesfor future work.6.6.4 Experiments and Results To determine the benefits of our FV cache architecture in reducing staticenergy, we ran 11 SPEC2000 [26] benchmarks through the SimpleScalartool set [19]. We used a 4-issue out-of-order processor simulator with a 32Kbyte L1 instruction and data cache. The benchmarks were fast-forwardedfor 1 billion instructions and executed for 500 million instructionsafterwards, using reference inputs. Static Energy Savings Our main goal is to reduce the static energy consumed by the data cachewithout losing performance. As mentioned earlier, the overall static energysaving depends on the average coverage of FVs inside data cache. Throughexperiments, we found that there are abundant FVs in the L1 data cache atany time for Spec 2000 benchmarks, as shown in Figure 6-5. The percentageshown is the average for the 500 million instructions execution time. Onaverage, 49.2% of the total words are FVs, with the highest being 77.0% forbenchmark mcf and the lowest 9.4% for benchmark ammp. The static energysavings are proportional to the number of FVs in the data cache. Thus, thecorresponding static energy savings on average are 35%(49.2%×27/33×86%) considering that 27 bits out of 33 bits (we need a flagbit per 32-bit word) are shut off and 86% of static power can be saved usinga pMOS Gated-Vdd. When compared with the conventional 32-bit per wordcache, the static energy savings can be calculated as 100%- (100%-35%)*33/32 = 33%.
  • 120 80% 60% 40% 20% 0% gzip ammp mcf bzip votex mesa Ave equake parser vpr gcc art Figure 6-5. Percentage of data cache words that are FVs Performance Improvement Our second achievement is the performance improvement over the original FV data cache design. Recall that the original FV cache performance overhead was due to the prolonged non-FV accesses. The more non-FV accesses, the slower the execution and the less the overall power savings (less energy savings), since the system would consume more energy when the program runs longer. We measured the average percentage of cache hits that are FVs, as shown in Figure 6-6(a). On average, the hit rate on data FVs is 32% with the highest being 62.7% for votex and the lowest 11.4% for mcf. Therefore, we can see that on average, 68% of cache accesses are non-FVs. 75% 2.0 Normal Cache 2-cy cle FVC 50% 1.5 IPC 25% 1.0 0.5 0% 0.0 gcc equake bzip gzip mesa Ave mcf parser vpr art ammp votex gcc equake gzip mesa bzip2 Ave mcf parser vpr art ammp vortex (a) (b) Figure 6-6. (a) Hit rate of FVs in data cache; (b) Performance (IPC) degradation of two-cycle FV cache With our improved circuitry (1-cycle latency for non-FVs as well as for FVs), we are able to maintain the same execution speed as the base case. To see how much performance we have gained over the original FV cache, we measured the IPCs for a normal cache and a 2-cycle FV cache and plot them in Figure 6-6(b). The IPC for our improved design is the same as the normal cache. Figure 6-6(b) shows the slowdowns of the original FV cache design, which is the same value as our performance improvement. We can see that there is a 5.2% difference in the averaged IPCs between the original FV cache and our improved version. This also means that in addition to the static energy we saved by shutting off partial FV words, we also saved more dynamic energy than the original FV cache design.
  • 121 Another feature in our new design is that it is safe in the sense that it doesnot increase power consumption significantly even when FVs are notabundant. Thus, our improved FV cache design is an appealing approach inreducing both static and dynamic energy of caches. Acknowledgements This work was supported by the National Science Foundation (CCR-0203829, CCR-9876006) and by the Semiconductor Research Corporation(2003-HJ-1046G). References[1] S. Segars. Low power design techniques for microprocessors, International Solid State Circuit Conference, February 2001.[2] C. Zhang, F. Vahid, and W. Najjar. A highly-configurable cache architecture for embedded systems. 30th Annual International Symposium on Computer Architecture, June 2003.[3] Altera, Nios Embedded Processor System Development, http://www.altera.com/corporate/ news_room/releases/products/nr-nios_delivers_goods.html.[4] Arc International, www.arccores.com.[5] ARM, www.arm.com.[6] MIPS Technologies, www.mips.com.[7] Tensilica, Xtensa Processor Generator, http://www.tensilica.com/.[8] D. H. Albonesi. Selective Cache Ways: On Demand Cache Resource Allocation. Journal of Instruction Level Parallelism, May 2002. [9] A. Malik, W. Moyer, and D. Cermak. A Low Power Unified Cache Architecture Providing Power and Performance Flexibility. International Symposium on Low Power Electronics and Design, 2000.[10] R. Balasubramonian, D. Albonesi, A. Buyuktosunoglu, and S. Dwarkadas. Memory Heirarchy Reconfiguration For Energy and Performance in General-Purpose Processor Architecture. 33rd International Symposium on Microarchitecture, December 2000.[11] A. Veidenbaum, W. Tang, R. Gupta, A. Nicolau, and X. Ji. Cache Access and Cache Time Model. IEEE Journal of Solid-State Circuits, Vol 31, No 5, 1996.[12] T. Givargis and F. Vahid. Platune: A Tuning Framework For System-On-a-Chip Platforms. IEEE Transactions on Computer Aided Design, November 2002.[13] M. Palesi and T. Givargis. Multi-Objective Design Space Exploration Using Genetic Algorithms. International Workshop on Hardware/Software Codesign, May 2002.[14] C. Zhang and F. Vahid. Cache Configuration Exploration on Prototyping Platforms. 14th IEEE International Workshop on Rapid System Prototyping , June 2003.[15] A. Ghosh and T. Givargis. Cache Optimization For Embedded Processor Cores: An Analytical approach. International Conference on Computer Aided Design, November 2003.[16] C. Zhang, F. Vahid, and R. Lysecky. A Self-Tuning Cache Architecture for Embedded Systems. Design Automation and Test in Europe Conference (DATE), February 2004.[17] M. Powell, A.Agarwal, T. Vijaykumar, B. Falsafi, and K. Roy. Reducing Set-Associative Cache Energy via Way-Prediction and Selective Direct Mapping, 34th International Symposium on Microarchitecture, 2001.
  • 122 [18] C. Lee, M. Potkonjak, and W.H. Mangione-Smith. MediaBench: A Tool For Evaluating and Synthesizing Multimedia and Communication Systems. Proc 30th Annual International Symposium on Microarchitecture, December 1997. [19] D. Burger, T. Austin, and S. Bennet. Evaluating Future Microprocessors: The Simplescalar Toolset. University of Wisconsin-Madison. Computer Science Department Tech. Report CS-TR-1308, July 2000. [20] http://www.mips.com/products/s2p3.html, 2003. [21] A. Gordon-Ross, F. Vahid, and N. Dutt. Automatic Tuning of Two-Level Caches to Embedded Applications. Design Automation and Test in Europe Conference (DATE), February 2004. [22] EEMBC, the Embedded Microprocessor Benchmark Consortium, www.eembc.org. [23] N. Jouppi. Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers, Proceedings of International Symposium on Computer Architecture, 1990. [24] G. Albera and R. Bahar. Power/performance Advantages of Victim Buffer in High- Performance Processors, IEEE Alessandro Volta Memorial Workshop on Low-Power Design, 1999. [25] C. Zhang and F. Vahid. Using a Victim Buffer in an Application-Specific Memory Hierarchy. Design Automation and Test in Europe Conference (DATE), February 2004. [26] http://www.specbench.org/osg/cpu2000. [27] J. Yang and R. Gupta. Energy Efficient Frequent Value Data Cache Design, Int. Symp. on Microarchitecture, Nov. 2002. [28] C. Zhang, J. Yang, and F. Vahid. Low Static-Power Frequent-Value Data Caches. Design Automation and Test in Europe Conference (DATE), February 2004. [29] J. Yang and R. Gupta. “Frequent Value Locality and its Applications,” ACM Transactions on Embedded Computing Systems (inaugural issue), Vol. 1, No. 1, pages 79-105, November 2000.
  • 123Chapter 7REDUCING ENERGY CONSUMPTIONIN CHIP MULTIPROCESSORS USINGWORKLOAD VARIATIONSI. Kadayif1, M. Kandemir2, N. Vijaykrishnan2, M. J. Irwin2 and I. Kolcu31 Canakkale Onsekiz Mart University;2 Pennsylvania State University;3 UMISTAbstract Advances in semiconductor technology are enabling designs with several hundred million transistors. Since building sophisticated single processor based systems is a complex process from design, verification, and software development per- spectives, the use of chip multiprocessing is inevitable in future microprocessors. In fact, the abundance of explicit loop-level parallelism in many embedded ap- plications helps us identify chip multiprocessing as one of the most promising directions in designing systems for embedded applications. Another architectural trend that we observe in embedded systems, namely, multi-voltage processors, is driven by the need of reducing energy consumption during program execution. Practical implementations such as Transmeta’s Crusoe and Intel’s XScale tune processor voltage/frequency depending on current execution load. Considering these two trends, chip multiprocessing and voltage/frequency scaling, this chapter presents an optimization strategy for an architecture that makes use of both chip parallelism and voltage scaling. In our proposal, the compiler takes advantage of heterogeneity in parallel execution between the loads of different processors and assigns different voltages/frequencies to different processors if doing so reduces energy consumption without increasing overall execution cycles significantly. Our experiments with a set of applications show that this optimization can bring large energy benefits without much performance loss.Keywords: Chip multiprocessing, voltage scaling, loop-level parallelism, embedded systems, optimizing compilers.
  • 1247.1 INTRODUCTION Rising development costs motivate computer architecture companies to de-sign fewer systems-on-chip, but to make each one they do design more flex-ible and programmable. Doing so makes it possible to reuse designs to takeadvantage of economies of scale and shorten time-to-market. Moreover, pro-grammability allows companies to keep products in the market longer, boostingintegrated profits. High-performance embedded processors have traditionally relied mainly onclock frequency and superscalar instruction issue to boost performance. Whilefrequency and superscalarity have served the industry well and will continueto be used, we believe that they have limitations that will diminish the gainsthey will deliver in the future. The gains in operating frequencies, which havehistorically come at a rate of about 35 percent per year, are attributable to twomajor factors: semiconductor feature scaling and deeper pipelining. But eachof these factors is approaching the point of diminishing returns. Similarly,superscalar processing is nearing its limits, mainly due to the exponential in-crease in complexity in dispatch logic with increasing issue width. In addition,superscalar processing is limited by the inherent instruction-level parallelismin the code. Although VLIW implementations are less complex than theirsuperscalar counterparts (since most of execution decisions are made by thecompiler), they still employ power-hungry components and are limited by theavailable instruction-level parallelism. It should also be noted that both super-scalar and VLIW architectures are not efficient from an energy consumptionviewpoint. Therefore, it is not clear whether current architectures will be suffi-cient for meeting continuously increasing power and performance demands ofapplications. These observations motive system designers to investigate different archi-tectures. When one looks at computer architecture industry today, two differenttrends in system design can easily be observed: on-chip multi-processing andmulti-voltage processors. On-chip multi-processors take advantage of high-level, coarse-grain parallelism that exists due to the natural independence ofseparate program fragments (e.g., functions and loops). As compared to super-scalar and VLIW architectures, they are much more suitable for array-intensiveembedded applications. Another advantage of using an on-chip multiprocessor,instead of a more powerful and sophisticated uniprocessor, is that there is lessdifficulty in designing a smaller, less complex chip. This also speedups chipverification and validation. Thus, time required to put the chip in the marketbecomes shorter. One can see several examples of on-chip multi-processingtoday in both academia and industry. For example, the four-core Hydra fromStanford University [14] is built around Integrated Device Technology Inc.’sRC32364 processor, which uses a 0.25-micron process, and runs at 250 MHz.
  • 125As manufacturing processes keep getting refined, it becomes even easier toreplicate the core several times on a single die. The MAJC architecture fromSun Microsystems [11] allows one to four processors to share the same die, andfor each to run separate threads. Each processor is limited to four functionalunits (each of which are able execute both integer and floating point operations,making the MAJC architecture more flexible). Another example of an on-chipmulti-processor from industry is the Power4 processor from IBM [15], wheretwo processors are placed into the same die. The second trend, multi-voltage processors, is mainly driven by the need toreduce energy consumption during program execution. Practical implementa-tions such as Transmeta’s Crusoe [10] and Intel’s XScale [8] scale processorvoltage/frequency depending on execution load. Observing that one rarelyneeds an application to exercise a processor’s maximum performance and theunused extra performance usually represents wasted energy, Crusoe designerstry to match the operating level of the processor (in terms of voltage and fre-quency) to the performance requirements of the application being executed.Depending on the voltage regulator, a Crusoe processor can change its voltagein steps of 25mV and its frequency in steps of 33MHz. Considering the continuously pressing power and performance demands, wecan expect these two techniques to be co-exist in the future embedded archi-tectures. Specifically, we believe that future architectures will be based onon-chip multi-processors, where each on-chip processor can be individuallyvoltage/frequency scaled. Considering such an architecture, this paper investi-gates the energy/performance tradeoffs in parallelizing array-intensive applica-tions taking into account the possibility that individual processors can operatein different voltage/frequency levels. In assigning voltage levels to processors,we make use of compiler analysis that reveals heterogeneity between the loadsof different processors in parallel execution. Our experiments with a set of ap-plications show that the proposed optimization can bring large energy benefitswithout much performance penalty. The rest of this chapter is organized as follows. The next sections describesour chip multiprocessor. Section 7.3 discusses why we may be experiencingload imbalance across on-chip processors at runtime. Section 7.4 discussesthe necessary compiler analysis for determining workloads (on a loop nest ba-sis) of individual processors participating in parallel computation. Section 7.5discusses additional optimizations to further enhance our power savings. Sec-tion 7.6 describes our implementation, experimental platforms, and presentsperformance and energy numbers. Section 7.7 presents our concluding re-marks.
  • 126 CPU 0 CPU 1 CPU 2 CPU 3 Cache Cache1 Cache 2 Cache 0 3 L2 Cache Optional Off-Chip Figure 7.1. Chip multiprocessor under consideration.7.2 CHIP MULTIPROCESSOR ARCHITECTURE AND EXECUTION MODEL The chip multiprocessor we consider here is a shared-memory architecture;that is, the entire address space is accessible by all processors. Each processorhas a private L1 cache, and shared memory is assumed to be off-chip. Option-ally, we may include a (shared) L2 cache as well. Note that several architecturesfrom academia and industry fit in this description [1, 14, 11, 12]. We keep thesubsequent discussion simple by using a shared bus as the interconnect (thoughone could use fancier/higher bandwidth interconnects as well). We also usethe MESI [19] protocol (the choice is orthogonal to the focus of this paper)to keep the caches coherent across the CPUs. We assume that voltage leveland frequency of each processor in this architecture can be set independentlyof the others, and this is the main mechanism through which we save power.This paper focuses on a single-issue, five-stage (instruction fetch (IF), instruc-tion decode/operand fetch (ID), execution (EXE), memory access (MEM), andwrite-back (WB) stages) pipelined datapath for each on-chip processor. Cur-rently, this is the only architectural model for which our compiler estimatesprocessor workload. Note that progress in VLSI technology has allowed chip-makers to packmillions of transistors in a single die. Rather than throwing all these resourcesinto a single, powerful processing core and making this core very complex todesign and verify, chip-multiprocessors consisting of several simpler proces-sor cores can offer a more cost-effective and simpler way of exploiting thesehigher levels of integration. Chip multiprocessors also offer a higher granu-larity (thread/process level) at which parallelism in programs can be exploitedby compiler/runtime support, rather than leaving it to the hardware to extractthe parallelism at the instruction level on a single (larger) multiple-issue core.All these compelling reasons motivate the trends toward chip multiprocessor
  • 127architectures, and there is clear evidence of this trend in the several commercialofferings and research projects [1, 14, 11, 12]. Our application execution strategy can be summarized as follows. We focuson array-based applications that are constructed from loop nests. Typically,each loop nest in such an application is small but executes a large number ofiterations and accesses/manipulates large datasets (typically multidimensionalarrays). We employ a loop nest based application parallelization strategy. Morespecifically, each loop nest is parallelized independently of the others. In thiscontext, parallelizing a loop nest means distributing its iterations across proces-sors and allowing processors to execute their portions in parallel. For example,a loop with 1000 iterations can be parallelized across 10 processors by allo-cating 100 iterations to each processor. We also assume that after each loopnest execution, all processors get synchronized before they start executing thenext loop nest. Note that dropping this requirement would necessitate a so-phisticated compiler analysis to identify the cases under which a processor thatfinishes its portion of iterations from the previous loop nest can go ahead andstart executing its portion from the next loop nest without waiting for the others.Nevertheless, in our experiments to be presented later, we also evaluate suchan alternative strategy. There are many proposals for power management of a dynamic voltagescaling-capable processor. Most of them are at operating system level andare either task-based [13, 17] or interval-based [21, 5]. While some proposalsaim at reducing energy without compromising performance, a recent study byGrunwald et al [6] observed noticeable performance loss for some interval-based algorithms using actual measurements. The existing compiler basedstudies such as [7, 16] target single processor architectures. In comparison, ourwork targets at a chip multiprocessor based environment.7.3 LOAD IMBALANCE IN PARALLEL EXECUTION We can broadly divide loop nest parallelization techniques into two cate-gories: static and dynamic. In the static case, the compiler (or the user) decidesa suitable parallelization strategy for each loop nest at compile time. The ideais to assign each loop iteration to a processor. There are at least two ways ofdoing this. In block assignment, a group of consecutive loop iterations are as-signed to the same processor. Since such iterations typically access data storedin consecutive memory locations, this type of assignment can also be expectedto be data locality friendly. In cyclic assignment, the iterations assigned toprocessors are interleaved using some stride. While this type of assignmentis known to be good from a load balance viewpoint, it generally exhibits poordata locality. Consider, as an example, the loop nest shown below and the arrayreference in it:
  • 128 (a) (b) (c) 111111111111 000000000000 111111111111 000000000000 111111111111 000000000000 000000000000 111111111111 P0 11111111111 00000000000 000000000000 P 111111111111 P1 111111111111 000000000000 11111111111 00000000000 P 111111111111 0 000000000000 000000000000 111111111111 11111111111 00000000000 11111111111 00000000000 0 111111111111 000000000000 111111111111 000000000000 P2 111111111 000000000 111111111111 000000000000 P 000000000000 111111111111 111111111 000000000 111111111111 000000000000 111111111111 1 000000000000 P3 000000000000 111111111111 000000000000 111111111111 000000000 P 111111111 1 111111111 000000000 111111111111 000000000000 P 111111111111 000000000000 111111111111 000000000000 111111111111 000000000000 P0 111111 000000 P 111111111111 000000000000 2 P1 000000000000 111111111111 000000000000 111111 000000 000000000000 111111111111 111111111111 111111 000000 2 P2 111111 000000 111111111111 000000000000 111111111111 000000000000 1111 0000 111111111111 000000000000 111111111111 000000000000 1111 0000 000000000000 P 111111111111 111111111111 3 000000000000 P3 000000000000 111111111111 111111111111 000000000000 0000 P 1111 3 1111 0000 (d) (e) 11 00 11 11111111111 00000000000 00 11 V0 11111111111 00000000000 P 00 11 11111111111 00000000000 0 00 11 00 11111111111 00000000000 11 00 11 00 111111111 000000000 11 00 11 00 111111111 000000000 11 00 11 00 V1 000000000 P 111111111 1 11 00 11 00 11 00 11 00 111111111 000000000 11 00 11 00 11 00 111111 000000 11 00 11 00 11 00 11 V2 111111 000000 P2 11 00 11 00 00 11 111111 000000 11 00 11 00 00 11 00 111111 000000 11 00 11 00 11 11 00 11 00 11 00 00 11 11 00 11111 00000 11 00 11 00 00 11 00 11 00 V3 11111 00000 P 11 00 11 00 11 00 11 00 11 00 11111 00000 11 00 00000 3 11111 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 V0 > V1 > V2 > V3 P3 P2 P1 P0Figure 7.2. Different array accesses imposed by different iteration assignments (the array isassumed to be row-major). for i: 1..1024 for j: 1..1024 ..X[i,j].. Assuming that only the i-loop is parallelized across four processors (P0through P3 ), Figure 7.2(a) illustrates how array X is accessed by the processorswhen block iteration assignment is used. In this assignment, each processorexecutes 256 × 1024 iterations, and accesses a group of consecutive rows ofthe array as depicted in Figure 7.2(a). However, it is also possible to parallelizethis loop (i) by distributing its iterations cyclicly across processors using someregular stride. For example, we can give the first 128 × 1024 iterations to thefirst processor, the next 128 × 1024 to the second one and so on, and whenwe give its quota to the last processor, we can repeat the whole process (untilall loop iterations have been assigned) starting over with the first processor.Figure 7.2(b) shows how array X is accessed by the processors under this cycliciteration assignment scheme. Note that the cyclic iteration distribution is flexi-ble in the sense that it can work with any stride. For example, instead of using128 × 1024 iteration chunks, we could have easily used 16 × 1024 or even 1× 1024 iteration chunks. In comparison, in a dynamic parallelization strategy, the assignment of iter-ations to processors is performed dynamically during the course of executionby a central controller. Typically, this controller gives a new set of loop it-erations to a processor when that processor is done with executing its currentset of assigned iterations. While the dynamic strategy is expected to balance
  • 129the workloads of processors better than static strategies (as it can take run-time constraints into account), it also incurs a much higher runtime cost — interms of both execution cycles and power consumption — (as compared to thestatic parallelization schemes) since decisions regarding iteration assignmentsare made at runtime. Therefore, our focus in this study is on static loop nestparallelization. Consider now the following loop nest: for i: 1..1024 for j: i..1024 ..X[i,j].. While this loop nest is similar to the previous one considered above, there isone significant difference: the lower bound of the inner loop (j) is i (insteadof 1). Figure 7.2(c) shows how the four processors access the array in questionwhen block iteration assignment is employed. Clearly, there is a significantload imbalance across the processors. Assuming that each iteration of this loopnest has the same cost (in terms of execution cycles) and all processors shouldsynchronize following the execution of the nest, there is not any advantage forthe processors with the light load to finish their set of iterations as soon as pos-sible. Instead, they can delay their executions (by reducing their frequencies)and lower their voltages to save energy while making sure that their executiondoes not take more time than that of the processor with the largest load (op-erating with the highest voltage level). Figure 7.2(d) illustrates such a voltageassignment, assuming that V0 is the highest voltage level available. The workpresented in this paper performs such a voltage-to-processor assignment foreach loop nest of a given array-based application. In a sense, in our frame-work the job of the compiler is not just to decide which loop iterations shouldbe assigned to which processors but also which supply voltage/frequency eachprocessor needs to use. Our objective is to save as much power as possiblewithout incurring much performance penalty. At this point, someone might claim that it would be better in this case (Fig-ure 7.2(c)) to use cyclic assignment instead of block assignment as this wouldeliminate the load imbalance problem introduced by the latter to a large extent.However, this may not be a viable option in general. Consider, for example,the scenario depicted in Figure 7.2(e), where the direction of parallelization isreversed (due to data dependences for example). In this case, cyclic assignmentwould be very costly in terms of data locality (cache behavior), assuming thatthe array in question is stored as row-major. Considering the fact that off-chipmemory accesses are getting more and more expensive in terms of processorcycle times, one may not want to degrade data locality.
  • 1307.4 COMPILER SUPPORT As mentioned earlier, the compiler’s job in our setting is to assign not onlyiterations to processors but also come up with a suitable voltage level for eachprocessor. To do this, the compiler needs to estimate the workload of eachprocessor and match it with an appropriate voltage/frequency level. Withoutloss of generality, we assume that there are s voltage/frequency levels availableto the compiler. Our compiler-based approach proceeds as follows: • Parallelization Step. In this step, the compiler parallelizes an applica-tion in a loop nest basis. That is, each loop nest is parallelized independentlyconsidering the intrinsic data dependences it has. Since we are targeting achip multiprocessor, our parallelization strategy tries to achieve (for each nest)outer-loop parallelism to the best extent possible. In other words, we parallelizethe outermost loop (in the nest) that carries no data dependence. Our baselineresults are obtained using this parallelization strategy. Later in our experiments,we change our parallelization strategy to conduct a sensitivity analysis. • Processor Load Estimation. In this step, the compiler estimates theload of each processor in each nest. To do this, it performs two calculations:(a) iteration count estimation and (b) per-iteration cost estimation. Since inmost array-based embedded applications bounds of loops are known beforeexecution starts, estimating the iteration count for each loop nest is not verydifficult. The challenge is in determining the cost (in terms of execution cycles)of a single iteration (for a given loop nest). Since the processors employed inour chip multiprocessor are simple single-issue cores, our cost computation isclosely dependent on the number and types of the assembly instructions thatwill be generated for the loop body. Specifically, we associate a base executioncost with each type of assembly instruction. In addition, we also estimatethe number of cache misses. Since loop-based embedded applications exhibitvery good instruction locality (as they spend most of their execution cycleswithin loop nests and there are not too many conditional-if executions), wefocus on data cache and estimate data cache misses using the method proposedby Carr et al [2]. An important issue is to estimate (at the source level) whatassembly instructions will be generated for the loop body in question. Weattack this problem as follows. The constructs that are vital to the studied codesinclude a typical loop, a nested loop, assignment statements, array references,and scalar variable references within and outside loops. Our objective is toestimate the number of assembly instructions of each type associated with theactual execution of these constructs. To achieve this, the assembly equivalentsof several codes were obtained using our back-end compiler (a variant of gcc)with the O2-level optimization. Next, the portions of the assembly code werecorrelated with corresponding high-level constructs to extract the number andtype of each instruction associated with the construct. In order to simplify the
  • 131correlation process and to partially isolate the impact of instruction choice dueto low-level optimizations, the assembly instructions with similar functionalityand energy consumption are grouped together. For example, both branch-if-not-equal (bne) and branch-if-equal (beq) are grouped as a generic branchinstruction (denoted bra). To illustrate our parameter extraction process in more detail, we focus onsome specifics of the following example constructs. First, let us focus on aloop construct. Each loop construct is modeled to have a one-time overhead toload the loop index variable into a register and initialize it. Each loop also hasan index comparison and an index increment (or decrement) overhead whosecosts are proportional to the number of loop iterations (called trip count or trip).From correlating the high-level loop construct to the corresponding assemblycode, each loop initialization code is estimated to execute one load (lw) andone add (add) instruction (in general). Similarly, an estimate of trip+1 load(lw), store-if-less-than (stl), and branch (bra) instructions is associated with theindex variable comparison. For index variable increment (resp. decrement),2×trip addition (resp. subtraction) and trip load, store, and jump instructionsare estimated to be performed. Next, we consider extracting the number of instructions associated with ar-ray accesses. First, the number and types of instructions required to computethe address of the element are identified. This requires the evaluation of thebase address of the array and the offset provided by the subscript(s). Our cur-rent implementation considers the dimensionality of the array in question, andcomputes the necessary instructions for obtaining each subscript value. Com-putation of the subscript operations is modeled using multiple shift and addi-tion/subtraction instructions (instead of multiplications) as this is the way ourback-end compiler generates code when invoked with the O2 optimization flag.Finally, an additional load/store instruction was associated to read/write thecorresponding array element. Note that these correlations between high-levelconstructs and low-level assembly instructions are a first-level approximationfor our simple architecture and array-dominated codes with the O2-level op-timization and obtained through extensive analysis of a large number of codefragments. Based on the process outlined above, the compiler estimates iteration countfor each processor and per-iteration cost. Then, by multiplying these two, itcalculates the estimated workload for each processor. While this workloadestimation may not be 100% accurate, it allows the compiler to rank processorsaccording to their workloads and assign suitable voltage levels and frequenciesto them as will be described in the next item. As an example consider thesecond loop nest shown above, parallelized using 4 processors. Assuming thatour estimator estimates the cost of loop body as L instructions, the loads of
  • 132processors P0 , P1 , P2 , and P3 are 256 × 1024 × L, 256 × (1024-257+1) × L,256 × (1024-513+1) × L, and 256 × (1024-769+1) × L, respectively. • Voltage Assignment. In this step, the compiler first orders the proces-sors according to non-increasing workloads. After that, the highest voltage isassigned to the processor with the largest workload (the objective being not toaffect the execution time to the greatest extent possible). Then, the processorwith the second highest workload gets assigned to the minimum voltage levelVk available (where 1 ≤ k ≤ s) that does not cause its execution time to exceedthat of the processors with the largest workload. In this way, each processorgets the minimum voltage level (to save maximum amount of power) withoutincreasing overall parallel execution time of the nest (which is determined bythe processor with the largest workload). Continuing with the example above,suppose that we have two voltage/frequency levels (that is, V1 /f1 and V2 /f2 ,assuming s = 2 and V1 /f1 > V2 /f2 ), we first determine the execution time takenby processor P0 (denoted T0 ). Then, for each other processor, we use V2 /f2if doing so does not cause their execution times to exceed T0 . If any of theseexecution times exceeds T0 (when using V2 /f2 ), we switch back to V1 /f1 forthat processor. The success of our strategy critically depends on two important factors. First,there should be some load imbalance to exploit between different processors.This is because if there is no such imbalance then it is reasonable to executeeach processor with the highest voltage/frequency. Second, the compiler-basedworkload estimation should be reasonably accurate. If this is not the case, thenwe may assign a wrong voltage level/frequency to a processor, which may inturn impact overall execution time. In fact, in this scheme, the only time wepay some penalty is when our compiler-based workload estimation is not veryaccurate. In our experiments, we quantify this penalty in detail.7.5 ADDITIONAL OPTIMIZATIONS In this section, we discuss how the effectiveness of our strategy can be furtherincreased using additional optimizations.7.5.1 Inter-Nest Optimization In the description of our strategy above, we assumed that the processors willsynchronize at the end of each loop nest (before they start executing the nextloop nest). As noted by Tseng [20], such a global synchronization presentstwo major problems. First, to implement such a synchronization, the compilerneeds to generate extra (synchronization) code and insert it in the applicationcode. Obviously, this code presents extra performance and power overhead atruntime. Second, since this synchronization requires all processors to wait forthe slowest one, it makes poor use of available resources (from the performance
  • 133angle). Consequently, allowing a processor to continue without waiting forthe slower ones can allow small perturbations in processor execution times toeven out, thereby improving overall performance (by taking advantage of theloosely-coupled nature of chip multiprocessors). However, determining whenit is safe to allow a processor to continue without synchronization requires extracompiler analysis. In this study, we implemented a strategy that takes a number(called b) as a parameter, and for each loop nest, allows a processor to continuefor at most b next nests if doing so does not violate any data dependences.7.5.2 Voltage/Frequency Reuse Another optimization can be performed by being more careful in voltageassignment. Up to this point in our discussion we assumed that the processorassignment for each loop nest is done independently of the other nests. As aresult of this, as we move from one loop nest to another the same processor canget assigned different voltage levels. Consequently, we pay a penalty (in termsof both performance and energy consumption) for changing voltage levels. Thispenalty can be minimized by reusing the same voltage as much as possible forthe same processor throughout the execution. This can be achieved as follows.Suppose that in loop nest i, we used voltage level Vk for processor j. Whenwe move to loop nest i + 1 if we need to assign voltage level Vk to a processor,we use processor j for that. This can be repeated for each neighboring loopnest pair, and in this way, the processors reuse their voltage levels as much aspossible.7.5.3 Adaptive Parallelization So far in our treatment of the subject, we have assumed that we use allavailable processors in execution of all nests in the application. However, itis known from prior research [9] that, in some cases using fewer processors(and shutting off the unused ones along with their L1 caches) can result in abetter energy consumption behavior. We also conducted experiments with anadaptive strategy, where each loop nest is first profiled using different numberof processors in conjunction with our optimization strategy. After the profiling,for each loop nest, we identified the ideal number of processors, and used itin the actual execution. It should be noted that in adaptive parallelization weuse fewer number of processors than available (this means some performanceloss); however, turning off unused processors along with their L1 caches canbring energy benefits.7.5.4 Combining Cyclic and Block Iteration Allocations As has been discussed earlier in the paper, one may also opt to use cyclicdistribution of loop iterations across processors. Since our framework is able to
  • 134Table 7.1. Base simulation parameters used in our experiments. Parameter Default Value Number of Voltage/Frequency Levels 8 Lowest/Highest Voltage Levels 0.8V/1.4V Frequency Step Size 30MHz Voltage/Frequency Transition Penalty 10 cycles/2.10nJ L1 Size 8KB L1 Line Size 32 bytes L1 Associativity 4-way L1 Latency 1 cycle L2 Size (Shared) 2MB L2 Associativity 4-way L2 Line Size 64 bytes L2 Latency 10 cycles Memory Access Latency 100 cycles Bus Arbitration Delay 5 cycles Replacement Policy Strict LRU L1 Energy (per access) 1.14nJ L2 Energy (per access) 2.56nJ Main Memory Energy (per access) 23.10nJestimate the number of cache misses, we can potentially have a better strategyas follows. For each loop nest, we can calculate the number of misses for bothblock and cyclic wise allocations and select the strategy that generates the bestenergy savings under a performance (execution cycles) constraint. We can referto such a strategy as hybrid since it makes use of both block and cyclic wiseallocation.7.6 EXPERIMENTS We tested the effectiveness of our algorithm in reducing energy consump-tion of chip multiprocessor using six array-intensive programs: 3D, DFE, LU,SPLAT, MGRID, and WAVE5. 3D is an image-based modeling applicationthat simplifies the task of building 3D models and scenes. DFE is a digitalimage filtering and enhancement code. LU is an LU decomposition program.SPLAT is a volume rendering application which is used in multi-resolution vol-ume visualization through hierarchical wavelet splatting. Finally, MGRID andWAVE5 are C versions of two Spec95FP applications. These C programs arewritten in such a fashion that they can operate on inputs of different sizes. Thedefault configuration parameters used in our experiments are given in Table7.1, and these are the values that are used unless explicitly stated/varied in thesensitivity experiments. To conduct our experiments, we modified Simics [18]. Simics is a full systemsimulation platform that can simulate both uniprocessor and multiprocessor
  • 135Figure 7.3. Normalized energy consumption with different number of processors (8 voltagelevels).machines. All energy results reported in this section include the energy spentin CPUs, their caches, and main memory and have been normalized with respectto the energy consumption when no voltage scaling is used and each processoris operated with maximum supply voltage and frequency. The graph in Figure 7.3 gives the normalized energy consumptions withdifferent number of processors. We can make two main observations fromthis graph. First, all our six applications get some energy benefit from ourapproach with all processor sizes experimented. Second, our energy savingsget better with increased number of processors. This is because a larger numberof processors means more load imbalance to optimize, and our approach takesadvantage of it. When considering individual applications, one can see thatMGRID and WAVE5 perform poorly as compared to the others, mainly becausethese applications have very few cases where our approach is applicable. Incomparison, LU benefits much from increasing the number of processors sincemost of its few loops exhibit significant amount of load imbalance. Overall,the average savings across all six applications are between 16.03% (for the twoprocessor case) and 41.80% (for the thirty-two processor case). To evaluate theimpact of the number of voltage levels on energy savings, we also performedexperiments with different number of voltage levels. The results are presentedin Figure 7.4 for the 8 processor case. One can easily see from this graphthat the number of voltage levels has a significant impact on energy behavior.In particular, the difference in going from 4 levels to 8 levels is dramatic; thecorresponding savings are 6.63% and 29.02%. Increasing the number of voltagelevels further (to 16) does not bring too much additional energy benefits sincethere is little scope left to be optimized (beyond what could be optimized using
  • 136Figure 7.4. Normalized energy consumption with different voltage levels (8 processors).8 levels). It should also be mentioned that when we have only 2 levels, theaverage saving across all applications is only 2.40%. This poor results is dueto the fact that our strategy tries not to increase execution cycles as much aspossible. Consequently, in many cases (when we have only 2 voltage levels)the compiler cannot use the lower voltage for a processor (even though theprocessor has low workload) since doing so would increase execution cyclesdramatically. Recall that in Section 7.5 we discussed four different optimization strategiesthat can further increase energy savings. The graph shown in Figure 7.5 givesnormalized energy consumptions with these optimizations. The first bar foreach application corresponds to our strategy when none of these four optimiza-tions have been activated. Our first observation is that each application benefitsfrom one or more of these optimizations. Second, not every optimization is ef-fective for each benchmark. For example, using the hybrid iteration allocationbrings energy benefits in only 3D and DFE (since the nests in other allocationsexhibit a uniform behavior and prefer only one type of iteration allocation forthe best energy behavior). Similarly, adaptive parallelization is useful only for3D and SPLAT. To further study the impact of inter-nest optimization (one of theoptimizations discussed in Section 7.5), we also performed experiments withdifferent values for b (nore that the default value that we used in Figure 7.5 is 4).We see from the graph in Figure 7.6 that for the applications that benefit fromthis optimization, a b value of 4 seems to be reasonable. This is because in manycases the data dependences in the application prevent a processor from goingbeyond the next four nests to execute without waiting for the slower processors. While the energy savings reported in this section are significant, for a faircomparison one also needs to consider the impact of our approach on perfor-
  • 137Figure 7.5. Impact of different optimizations on energy consumption (8 processors; 8 voltagelevels; and b = 4).Figure 7.6. Impact of b on energy consumption (8 processors and 8 voltage levels).mance. As has been pointed out earlier, our approach can lead to an increase inexecution cycles only if the compiler analysis is largely inaccurate. The graphin Figure 7.7 shows that the performance overhead incurred by our approachis below 2% in all but one (SPLAT) application. The reason that we have arelatively large performance penalty in SPLAT is the fact that this applicationexhibits a large number of conflict misses (over 68%, and rest are cold and ca-pacity misses), which cannot be captured by the cache miss estimation schemecurrently employed by our implementation. Consequently, the compiler is notvery successful in attaching suitable voltage levels to processors, and this inturn causes performance degradation. It is conceivable that a more accurate
  • 138Figure 7.7. Percentage increase in execution cycles (8 processors; 8 voltage levels).cache miss estimation strategy (e.g., [4]) can help improve the behavior of thisbenchmark. This will be part of our future research on this topic.7.7 CONCLUDING REMARKS A chip multiprocessor lowers the number of functional units per processor,and distributes separate tasks/threads to each processor. This paper has evalu-ated a compiler-directed strategy that allows different processors to use differentvoltage levels/frequencies to take advantage of the load imbalances stemmingfrom loop parallelization. Our results with six applications clearly demonstratethe effectiveness of our strategy and makes a case for voltage-sensitive loopparallelization. Our results also show that it is possible to increase energysavings further by employing voltage/frequency reuse, adaptive parallelization,and inter-nest optimization.Acknowledgments This work was supported in part by NSF Career Awards #0093082 and#0093085, and a grant from GSRC PAS.References[1] L. A. Barroso, K. Gharachorloo, R. McNamara, A. Nowatzyk, S. Qadeer, B. Sano, S. Smith, R. Stets, and B. Verghese. Piranha: A Scalable Architecture Based on Single-Chip Multiprocessing. Proceedings of International Sym- posium on Computer Architecture, Vancouver, Canada, June 12–14 2000.[2] S. Carr, K. S. McKinley, and C. Tseng. Compiler Optimizations for Im- proving Data Locality. Proceedings of the Sixth International Conference
  • 139 on Architectural Support for Programming Languages and Operating Sys- tems, San Jose, October 1994.[3] DAC’02 Sessions: Design Methodologies Meet Network Applications and System on Chip Design, New Orleans, LA, June 2002.[4] S. Ghosh, M. Martonosi, and S. Malik. Cache Miss Equations: An An- alytical Representation of Cache Misses. Proceedings of the 11th ACM International Conference on Supercomputing, July, 1997.[5] K. Govil, E. Chan, and H. Wasserman. Comparing Algorithms for Dynamic Speed-Setting of a Low-Power CPU. Proceedings of the 1st ACM Interna- tional Conference on Mobile Computing and Networking, November 1995.[6] D. Grunwald, P. Levis, K. Farkas, C. Morrey III, and M. Neufeld. Poli- cies for Dynamic Clock Scheduling. Proceedings of the 4th Symposium on Operating System Design and Implementation, October 2000.[7] C.-H. Hsu and U. Kremer. Dynamic Voltage and Frequency Scaling for Scientific Applications. Proceedings of the 14th Workshop on Languages and Compilers for Parallel Computing, August 2001.[8] Intel XScale Technology. http://www.intel.com/design/intelxscale/.[9] I. Kadayif, M. Kandemir, and U. Sezer. An Integer Linear Programming Based Approach for Parallelizing Applications in On-Chip Multiproces- sors. In Proc. Design Automation Conference, New Orleans, LA, June 2002.[10] A. Klaiber. The Technology Behind Crusoe Pro- cessors. Transmeta White Paper, January 2000. http://www.transmeta.com/about/press/white papers.html.[11] MAJC-5200. http://www.sun.com/microelectronics/MAJC/5200wp .html[12] MP98: A Mobile Processor. http://www.labs.nec.co.jp/MP98/top-e.htm.[13] T. Okuma, T. Ishihara, and H. Yasuura. Real-Time Task Scheduling for a Variable Voltage Processor. Proceedings of the 12th International Sympo- sium on System Synthesis, 1999.[14] K. Olukotun, B. A. Nayfeh, L. Hammond, K. Wilson, and K. Chang. The Case for a Single Chip Multiprocessor. Proceedings of the 7th Intl Confer- ence on Architectural Support for Programming Languages and Operating Systems, ACM Press, New York, 1996, pp. 2–11.[15] POWER4 System Microarchitecture, White Paper, http://www-1.ibm .com/servers/eserver/pseries/hardware/whitepapers/power4 .html[16] H. Saputra, M. Kandemir, N. Vijaykrishnan, M. J. Irwin, J. S. Hu, C- H. Hsu, and U. Kremer. Energy-Conscious Compilation Based on Voltage
  • 140 Scaling. Proceedings of ACM SIGPLAN Joint Conference LCTES’02 and SCOPES’02, Berlin , Germany, June, 2002.[17] Y. Shin, K. Choi, and T. Sakurai. Power Optimization of Real-Time Em- bedded Systems on Variable Speed Processors. Proceedings of the Inter- national Conference on Computer-Aided Design, November 2000.[18] SIMICS. http://www.virtutech.com/simics/simics.html.[19] J. P. Singh and D. Culler. Parallel Computer Architecture: A Hardware- Software Approach, Morgan-Kaufmann, 1998.[20] C.-W. Tseng. Compiler Optimizations for Eliminating Barrier Synchro- nization. Proceedings of 5th ACM Symposium on Principles and Practice of Parallel Programming, Santa Barbara, CA, July 1995.[21] M. Weiser, B. Welch, A. Demers, and S. Shenker. Scheduling for Reduced CPU Energy. Proceedings of the 1st Symposium on Operating Systems Design and Implementation, November 1994.
  • 141Chapter 8ARCHITECTURES AND DESIGN TECHNIQUESFOR ENERGY EFFICIENT EMBEDDED DSPAND MULTIMEDIA PROCESSINGIngrid Verbauwhede1,2, Patrick Schaumont1, Christian Piguet3, BartKienhuis41 University of California, Los Angeles; 2K.U.Leuven; 3CSEM; 4LeidenAbstract Energy efficient embedded systems consist of a heterogeneous collection of very specific building blocks, connected together by a complex network of many dedicated busses and interconnect options. The trend to merge multiple functions into one device makes the design and integration of these “systems- on-chip” (SOC’s) even more challenging. Yet, specifications and applications are never fixed and require the embedded units to be programmable. The topic of this chapter is to give the designer architectures and design techniques to find the right balance between energy efficiency and flexibility. The key is to include programmability (or reconfiguration) at the right level of abstraction and tuned to the application domain. The challenge is to provide an exploration and programming environment for this heterogeneous architecture platform.Keywords: Embedded systems, architectures, low power, design tools, design exploration8.1 INTRODUCTION Embedded systems (e.g. a cell phone, a GPS receiver, a portable DVDplayer, a HDD camcorder) use an architecture that is a heterogeneouscollection of very specific building blocks, connected together by a complexnetwork of many dedicated busses and interconnect options. General-purpose programmable processors are not used for energy efficiencyreasons. Typically, multiple small embedded processor cores withaccelerators, IP cores, etc. are used. The trend to merge multiple functions
  • 142 into one device (e.g. a cell phone with video capabilities) makes the design and integration of these “systems-on-chip” (SOC’s) even more challenging. Yet, specifications and applications are never fixed and require the embedded units to be programmable. A good balance between energy efficiency and programmability can be obtained by using programmable domain-specific processors. A well known example are the programmable digital signal processors (DSPs). DSPs are developed for wireless communication systems (mostly driven by cellular standards). In a first generation this meant that DSPs were adapted to execute many types of filters (e.g. FIR, IRR), later communication algorithms such as Viterbi decoding and more recently Turbo decoding are added. A first trend we notice is that more applications and multiple applications run in parallel or on demand on the device, e.g. video decoding, data processing, multiple standards, etc. A second trend we notice is that these new applications tend to run either on a separate domain specific programmable processor or on a hardware accelerator (the distinction between the two being rather blurry) next to the embedded DSP or micro- controller instead of being tightly coupled into the instruction set of the host processor. A third trend we notice is that general-purpose programming environments are getting more heterogeneous and domain-specific. The general-purpose solutions are for energy efficiency reasons augmented with domain specific units, accelerators, IP cores, etc. This is clearly visible in FPGA’s, as the new generations now include specialized blocks such as embedded core’s, block RAM’s and large numbers of multipliers. One successful example is the Virtex-Pro family of Xilinx [17]. These devices contain up to four Power PC cores, multiple columns of SRAM, multiple columns of multipliers, Gbits IO transceivers, etc. The architecture design of this heterogeneous SOC is a search in a three dimensional design space, which we call the reconfiguration hierarchy [12]. First in the Y direction: at what level of abstraction should the programming be introduced? Secondly in the X direction: which component of the architecture should be programmable? Thirdly in the Z direction: what is the timing relation between processing and the configuration/programming? Programming can be introduced at multiple levels of abstraction. When it is introduced at the instruction set level, it is called a “programmable processor”. When it is introduced at the CLB level of an FPGA, it is called a reconfigurable device. Regarding components, a processor has four basic components: data paths, control, memory and interconnect. One has a choice of making some or all of them programmable. Then the third question is to compare the processing activity to the binding time. It makes a system configurable, reconfigurable, or dynamic reconfigurable.
  • 143 The challenge is to develop a design environment to navigate in this threedimensional design space. Several SOC platforms have been presented in literature. Most of themfocus on general -purpose regular architectures, e.g. [2]. Very few focus onthe low power issue and the need to tune the architecture towards theapplication. One example is the low power Maya platform [18]. Unique toour design approach is that we combine the design and programming of thearchitecture with an environment to explore the best options. The chapter is organized as follows. Section 8.2 and 8.3 look at thearchitecture design, while section 8.4 and 8.5 discuss the design exploration,co-design and co-simulation challenges.8.2 ENERGY EFFICIENT HETEROGENEOUS SOC’S The system designer needs an architecture platform that gives him thelowest energy consumption, but at the same time provides enough flexibilityto allow re-programming or re-configuration. The key to energy efficiency isto tune the architecture to the application domain. This means freezingflexibility in the X (components) and Y (level of abstraction) direction of thereconfiguration hierarchy. A hierarchy of so-called “Y charts” allows us todo this in a top-down fashion [5]. A complex SOC will consist of multiple domain specific processingengines. Each processor is programmable to a more or less degree. It can behighly programmable if the processor is a micro-controller or a DSP engineor a blank box of CLB units. The efficiency goes up as domain specificinstructions are added. An example of this is the addition of a MACinstruction to a DSP processor. Loosely coupled co-processors will be moreenergy efficient but less flexible as they fit a narrower application domain.An example is the Turbo coder acceleration unit. The ultimate energyefficient block is the optimized hard IP unit. Yet, it does not provide anyflexibility. In SOC a range and collection of these blocks are used. Similarly arguments can be made for the interconnect component of aSOC. Currently, we see only two extreme options: either dedicated one-to-one connections and specialized busses, which have the lowest powerconsumption (to a first order) or general-purpose global busses or inter-connect, as provided by FPGA’s [17] or networks on chip [2]. The latter twoare both general-purpose solutions at different levels of abstraction to givethe designer a maximum flexibility and programmability.
  • 144 Software Standard Protocol Algorithm Architecture Domain - MicroArchitectre Specific Circuit Hardware Networking Video Security Signal Proc MEMORY RF Video Crypto Signal Baseband Engine Processing CPU Processing Reconfigurable Interconnect Figure 8-1. Example RINGS Architecture. The proposed RINGS architecture [16] is an architecture platform that gives the designer the option to explore the energy flexibility trade-offs. An example is shown in Fig. 8-1. A RINGS architecture contains a heterogeneous set of building blocks: programmable cores, both DSP’s and micro-controllers, programmable and/or reconfigurable hardware accelerator units, specialized IP building blocks, front-end blocks, and so on. When designing a solution based on RINGS, it is important that the domain expert has freedom to select the appropriate level of flexibility, ranging from fully programmable approaches, such as embedded micro controllers or FPGA blocks to highly optimized IP blocks. For different domains, the flexibility will be supported in different ways as domains have different characteristics. This domain specific flexibility can be expressed as a do-main specific abstraction pyramid as shown for Networking, Video, and Signal Processing on Fig. 8-1. In case of Video, the engine will consist of elements expressed in the Video pyramid, for example dedicated co-processors. The SOC is connected together at the top level by a supervising software program, which typically runs on an embedded micro-controller. At the bottom level, the reconfigurable interconnect glues it together. The programming paradigm used in RINGS is a reconfigurable network-on-chip. Also in this network, flexibility can be traded for energy efficiency at different levels of abstraction. Designers can instantiate an arbitrary network of 1D and 2 D router modules leading to an architecture illustrated in Fig. 8-2.
  • 145 Proc A Proc B Proc A 2D 1D 2D router router router 2D 2D router router Proc X Proc Y Figure 8-2. Example of Network-on-chip. This network illustrates the three binding time concepts. At the level ofconfiguration, the static network architecture with routers is instantiated.Reconfiguration is done by means of reprogramming the routing tables andprogramming by giving each packet a target address. A traditionalreconfiguration is obtained by reprogramming the routing tables in eachnode. An alternative approach is to use an easy to reconfigure physicalchannel. One example of this is a CDMA based reconfigurable interconnect[6][16]. Fig. 8-3 shows a conceptual picture of a source-synchronous CDMAimplementation. Each sender and receiver gets a unique spreading code. Bychanging the Walsh code, a different configuration is obtained. Traditionalbusses, which are a TDMA channel, require hardware switches forreconfiguration. CDMA interconnect has the advantage that reconfigurationcan occur “on-the-fly.” ... MOD1 MOD3 ... X X X Figure 8-3. Reconfigurable Interconnect (a) TDMA (b) SS-CDMA Bus Interface [1].
  • 146 8.3 ULTRA LOW POWER COMPONENTS The focus of this section is on the architecture design options to design ultra low power processor components, in many cases without losing performance. DSP processors have real-time constraints or need to maximize their throughput for a given task while at the same time minimize the power or energy consumption. Therefore, the design of DSP processors is very challenging, as it has to take into account contradictory goals: an increased throughput request at a reduced energy budget. On top there are new issues due to very deep submicron technologies such as interconnect delays and leakage. For instance, hearing aids used analog filters 15 years ago and were designed as digital ASIC-like circuits 5 years ago. Today they are designed with powerful DSP processors below 1 Volt and 1 mW of power consumption [8]. Hearing aids companies require DSP processors just because they require flexibility, i.e. to program the applications in-house. The design of ultra-low power DSP cores has to be performed at all design levels, i.e. system, architecture, circuit and technology levels. We will focus in this section to DSP architectures, but VHDL implementations as well as cell libraries are important too. Latch-based implementations including gated clocks described in VHDL or Verilog, low-power standard cell libraries and leakage reduction circuit techniques are necessary to reduce power consumption at these low levels. Various DSP architectures can be and have been proposed to reduce significantly the power consumption while keeping the largest throughput. Beyond the single MAC DSP core of 5-10 years ago, it is well known that parallel architectures with several MAC working in parallel allow the designers to reduce the supply voltage and the power consumption at the same throughput. It is why many VLIW or multitask DSP architectures have been proposed and used even for hearing aids. The key parameter to benchmark these architectures is the number of simple operations executed per clock cycle, up to 50 or more. However, there are some drawbacks. The very large instruction words up to 256 bits increase significantly the energy per memory access. Some instructions in the set are still missing for new better algorithms. Finally the growing core complexity and transistor count becomes a problem because leakage is roughly proportional to the transistor count. To be significantly more energy efficient, there are basically two ways, however impacting either flexibility or the ease of programming: 1. To design specific very small DSP engines for each task, in such a way that each DSP task is executed in the most energy efficient way on the smallest piece of hardware [9]. For N DSP tasks within a given
  • 147 application, the resulting architecture will be N co-processors or hardware accelerators around a controller or a simple DSP core as illustrated on Fig. 8-1. Memory Memory Memory Memory x + - x Figure 8-4. Hardware Reconfiguration Example [3].2. To design reconfigurable architectures such as the DART cluster [3], in which configuration bits allow the user to modify the hardware in such a way that it can much better fit to the executed algorithms. Fig. 8-4 shows an example. Option 1 is definitively the best one regarding power consumption. EachDSP task uses the minimal number of transistors and transitions to performits work. The control code unavoidable in every application is alsoefficiently executed on the controller or on the simple DSP, and someunexpected DSP tasks can be executed on the simple DSP if no accelerator isavailable. However, the main issue is the software mapping of a givenapplication onto so many heterogeneous processors and co-processors (seeSection 4). Transistor count could be high and some co-processors fullyuseless for some applications. Regarding leakage, unused engines have to becut off from the supply voltages, resulting in complex procedures tostart/stop them. Reconfigurable DSP architectures are much more power efficient thanFPGAs. The key point is to reconfigure only a limited number of unitswithin the DSP core, such as some execution units and addressing units [11].The latter are interesting, as the operands fetch from memory is generally asevere bottleneck in parallel machines for which 8-16 operands are requiredeach clock cycle. So, sophisticated addressing modes can be dynamicallyreconfigured depending on the DSP task to be executed. Fig. 8-5 shows anexample in which several addressing modes can reconfigured depending onthe user’s algorithms. This AGU (Address Generation Unit) contains 4 indexregisters (a0 to a3), 4 offset registers (o0 to o3) and 4 modulo registers (m0
  • 148 to m3). All these registers could be used to generate a given addressing mode and to compute AGU registers updates. The VLIW AGU operation register (AGUOP) is controlled by an AGU reconfiguration register (i0 to i3) that could be reconfigured at any time and allows the programmer to generate new addressing modes. Fig. 8-5 shows two examples of AGU computations. In the first example, register i0 contains configuration data such as the multiplexers and the PREAD adder are configured to generate address a0 + (02>>1), while at the same time registers a1, a3 and o3 are updated with new values computed through POSAD1, POSAD2 and PREADR ALUs. The POSAD1 ALU is used to generate WP1 = (a1+o3) modulo m2, while the POSAD2 ALU is used to generate WP2= m3 + 02<<2, and the result of PREADR is used to update register a0. The second example (i2) generates WP2 that uses both POSAD1 and POSAD2 ALUs connected in series. The operation (ao-02)%m0 is performed in the POSAD1 ALU, while adding 03 is performed in the POSAD2 ALU. This flexibility allows the programmer to generate very complex addressing modes that cannot be available in conventional DSP cores with addressing modes only defined in their instruction sets. VLIW AGU Reconfigurable instruction registers WP1 WP2 WP3 I A O M i0 a0 o0 m0 WP1WP2 RP1 RP2 RP3 RP4 RP5 RP6 RP7 PREAD POSAD1 P2A P2B POSAD2 SELAGUOP i1 a1 o1 m1 i2 a2 o2 m2 i3 a3 o3 m3 RP1 RP2 RP3 RP4 RP5 in = AGUOP (n=0..3) POSAD1 PREAD Examples of in operations: i0: DM ADDR = a0+(o1>>1), RES1 RP6 RP7 RES2 WP1: a1 = (a1+o3)%m2, WP2: o3 = m3 + o2<<2 WP3: a0 = a0+(o1>>1), P2A P2B i2: DM ADDR = a2+o1, AGUOP WP1: none WP2: a0 = (a0-o2)%m0+o3 WP3: a2 = a2+o1 POSAD2 DM ADDR Figure 8-5. Addressing Modes Reconfiguration Example (MACGIC DSP). However, the power consumption is necessarily increased due to the relatively large number of reconfiguration bits that have to be loaded in the
  • 149configuration registers. Similarly, the reconfigurable units are necessarilymore complex that non-reconfigurable units in terms of transistor count andtherefore consume more. Software issues are also difficult, as users candefine new instructions or new addressing modes that are difficult to supportby the development tools.8.4 DESIGN & ARCHITECTURE EXPLORATION The way a system behaves depends on the architecture, the way theapplications are written, and how these applications are mapped onto thearchitecture as compactly expressed by the Y-chart [5]. Examples ofarchitectures for low-power have already been given in other sections. Onsuch architecture, mapping is typically done in case of reconfigurable fabricsby the behavioral synthesis tool and the place and route tools. In case ofDSPs and CPUs, the mapping is typically performed by C-compilersdedicated to a particular type of DSP or CPU. An important questionremains: how to specify the applications that they can take advantage of thearchitecture in an effective manner. A low-power architecture will typically employ different levels ofparallelism like bit-level parallelism, instruction parallelism or task-levelparallelism to take advantage of voltage scaling as already explained in theprevious section. To successfully map a DSP application at a high level, theapplications need to express task-level parallelism. This parallelism istypically not present, as the applications are written in sequential languageslike C or Matlab. Therefore, mapping them is often a manual process that isvery tedious and time consuming, leading to a sub optimal system. A designer would like to have tool support that converts automaticallythe sequential specification into a parallel format. Moreover, the tool shouldallow him to ‘play’ with the amount of parallelism extracted from thespecification. In general, such tools are lacking in embedded system design.Some companies, like Pico and Art (ARM/Adelante) try to provide limitedcommercial solutions but this field is still very much subject to research. TheCompaan tool suite [13] aims at providing designers the option to play withparallelism for applications that are so-called “Nested Loop Programs”, avery natural fit for DSP applications. A DSP application is specified in asubset of Matlab and is automatically converted by Compaan into a networkof parallel processes. These processes can be specified in “C’ and mapped,using a conventional C compiler, onto a DSP or CPU. On the other hand,they can also be specified in VHDL and mapped using the appropriate toolsonto some reconfigurable fabric or realized as a dedicated IP core [19].Hence, “programming” the RINGS architecture is reduced to putting some
  • 150 processes onto the CPUs and DSPs while others are mapped onto FPGAs or use dedicated IP cores. There are many ways we can find parallelism in the application and in the way we partition the processes of the CPUs, DSPs and reconfigurable resources. Being able to explore these options early on in the design phase is crucial to get efficient embedded low-power systems. To allow designers to do this exploration, Compaan is equipped with a suite of techniques [14] like Unfolding, Skewing and Merging, to allow designers to play with the level of parallelism exposed in the derived network of processes. Skewing and Unfolding increase the amount of parallelism, while Merging reduces parallelism. By performing these techniques, many different networks can be created that can be mapped in different ways onto the architecture. When applied in a systematic way, the design space can be explored and the best performing network of processes can be picked. The difference in utilization of the architecture for a particular network can be huge. By rewriting a DSP application (like Beam-forming) using the presented techniques, we are able to achieve performances on a QR algorithm (7 Antenna’s, 21 updates) ranging from 12MFlops to 472MFlops. We realized QR using commercial floating point IP cores from QinetiQ, which include pipelined 55 (Rotate) and 42 (Vectorize) stages. We achieved this performance increase without doing anything to the architecture or mapping tools, but only by playing with the way the QR application is written, effectively improving the way the pipelines of the IP cores are utilized. Using a system like Compaan, an experienced designer should be able to obtain very different performing networks in days, having the opportunity to explore different systems and picking the one that uses the least amount of power. 8.5 DOMAIN-SPECIFIC CO DESIGN ENVIRONMENTS As discussed in the previous section, parallelism and distributed processing are key to energy efficient architectures. Because the ensemble of architecture elements (processors, busses, memories) cooperate towards a common application, the designer faces a considerable co-simulation and co- design problem. A key requirement is to have a good design model. Such a model allows building of simulation tools, compilers and code generators. We will look at a highly successful design model for programmable systems: the instruction-set architecture (ISA). Next we will consider the approach taken by the RINGS architecture. In a classic Von-Neumann architecture, the instruction-set-architecture (ISA) model maintains a single, consistent and abstracted view to the
  • 151operation of the system. Such a view ties four independent architectureconcepts together: control, interconnect, storage, and data operations [15].This way the ISA becomes a template for the underlying target architecture,for which compiler algorithms (scheduling etc) can be developed. Oftenhowever, the ISA is unable to offer the right target template – in terms ofparallelism, storage capabilities or other. In the RINGS architecture, we do not use an ISA as an intermediatedesign model, but approach each of the four components that make up anISA independently. We enumerate them below and look at the requirementsthey impose on co-simulation and co-design.• Data Operations: Energy efficient operation requires us to specialize each operator as much as possible. A RINGS system contains multiple processing cores. These can include hardwired or programmable (DSP or RISC) processors. We thus need to be able to combine instruction-set simulation with hardware simulation.• Storage: Energy efficient operation requires us to distribute storage. In addition to the high-level design transformations discussed in the previous section, we target to minimize storage bandwidth and use multiple distributed memories. Each processor in RINGS will work inside of a private memory space. Many operations in multimedia can be implemented with dedicated storage architectures that take only a fraction of the energy cost of a full-blown ISA. Examples are matrix transposition or scan-conversion. Such dedicated storage can be captured as a hardwired processor.• Interconnect: The energy efficient interconnect architecture discussed in section 2 requires explicit expression of interconnect operations – in contrast to an ISA where this is implicitly encoded in the instruction format. A network-on-chip can be modeled as a dedicated hardware architecture [1]. On top of the network-on-chip a suitable network protocol must be implemented, for example message-passing with the MPI standard [7]. However, also this protocol is subject to specialization and/or hard-coding. For example, a hardwired DCT coding unit attached to a DSP core through RINGS will have a fixed communication pattern. This pattern can be hard-coded in a collapsed and optimized protocol stack.• Control: Energy efficient operation requires us to split the data-flow and control-flow in a RINGS architecture and handle them independently. Fig. 8-6 clarifies this point. It shows the effect of moving an AES encryption operation gradually from high-level software (Java) implementation to dedicated hardware implementation, while at the same time maintaining the interface to the high level Java model. It can be seen that the interface overhead goes from 0.8% for a C-accelerated AES to
  • 152 8000% for a hardware-accelerated AES! This overhead obviously is caused by all the interfaces moving data from Java to C to hardware and back. With the MPI message passing scheme, we have the freedom to route control flow and a data flow independently as messages. This way, we can eliminate or minimize this interface overhead. Java Rijndael Interface cycles 301,034 367 Interface 892 C Rijndael cycles ac 44,063 c ele rat Co-processor ion Rijndael cycles 11 Total Cycles 301, 034 44,430 903 Figure 8-6. Overhead of Tightly Coupled Data/Control Flow. When we put the elements together, we conclude that the RINGS co- design environment should accommodate multiple instruction-set simulators with user-specified hardware models. All of these must be embedded in a model of an on-chip network. The timing accuracy of the simulation should be precise enough to simulate interactions such as network-on-chip communication conflicts. On the other hand, the simulation must also be fast enough to support reasonable design exploration capabilities. We have built the ARMZILLA environment to evaluate one class of RINGS architectures, namely those that can be built with one or more ARM cores, a network-on-chip, and dedicated hardware processors. Fig. 8-7 illustrates the ARMZILLA setup. There are three components: a hardware simulation kernel (GEZEL), one or more instruction-set simulators (ISS), and a configuration unit. The GEZEL kernel [4] captures hardware models with the FSMD (Finite-State-Machine with Datapath) model-of- computation. It uses a specialized language and a scripted approach to promote interactive design exploration. The cycle-true models of GEZEL can also be automatically converted to synthesizable VHDL. For the ARM ISS we use the cycle-true SimIT-ARM environment [10]. The ARM ISS uses memory-mapped channels to connect to the GEZEL hardware models. Finally, the configuration unit specifies a symbolic name for each ARM ISS, and associates each ISS with an executable. This way the memory-mapped communication channels can be set up, and the hardware GEZEL models can address each ARM memory space uniquely.
  • 153 C C Network Hardware C On Chip Processors FDL FDL Cross Compiler EXE EXE Config EXE Configuration Unit ARM ISS GEZEL ARM ISS ARM ISS Kernel Memory-mapped ARMZILLA Channels VHDL Figure 8-7. The ARMZILLA Design Environment for ARM-based RING Processors. An example of what can be done with the ARMZILLA environment isshown in Table 8-1. This table shows cycle counts that were obtained afterpartitioning a JPEG encoding algorithm. The reference implementation runson a single-ARM ISS model. In the second implementation, we separate thechrominance and luminance channels over two ARM processors. This seemsa logical partition that splits the data operations roughly in two parts. But, italso creates a communication bottleneck in the on-chip network and theresulting implementation becomes slower then the O3-level optimizedsingle-processor implementation. The third implementation shows a betterpartitioning. In this case, the data streams are routed out of the ARM andinto dedicated hardware processors for JPEG encoder subtasks. Theseprocessors can communicate directly amongst themselves. All these simulations are cycle-accurate yet they can run efficiently. Forthe H.264 decoding on a dual ARM with network-on-chip for example,ARMZILLA offers a simulation speed of 176K cycles per second. Thesimulation speed varies with the complexity of the hardware model used. Asingle, stand-alone SimIT-ARM simulator runs at 1 MHz cycle-true on a3GHz Pentium.
  • 154 Table 8-1. Multiprocessor JPEG Encoding Performance Partition Cycle count 64x64 block One single ARM 1.223 M Dual ARM using split chrominance/ 1.336 M luminance channels Single ARM with color conversion, 313K transform coding, huffman coding as stand- alone hardware processors 8.6 CONCLUSIONS In this chapter, we presented architecture design and design exploration for low power systems-on-chip. Low power is obtained by tuning all components of the architecture (datapaths, control, memory and interconnect) to the application. This can occur at different levels of abstraction. The design of this type of SOC requires support by design models and methods. The design environments Compaan and Gezel /Armzilla are illustration of supporting tools for this design space exploration. References [1] D. Ching, P. Schaumont, I. Verbauwhede, “Integrated Modeling and Generation of a Reconfigurable Network-On-Chip,” Proc. 11th Reconfigurable Architectures Workshop, RAW 2004, Santa Fe, NM, April 2004. [2] W. Dally, B. Towles, “Route Packets, not wires: on-chip interconnection networks,” Proc. DAC 2001. [3] R. David et al., “Low-Power Reconfigurable Processors”, Chapter 20 in “Low Power E Electronics Design,” edited by C. Piguet, CRC Press, 2004. [4] GEZEL kernel, http://www.ee.ucla.edu/~schaum/gezel [5] B. Kienhuis, et al.,``A Methodology to Design Programmable Embedded Systems, LNCS, Vol 2268, Nov. 2001. [6] J. Kim, et al., “A 2-Gb/s/pin Source Synchronous CDMA Bus Interface with simultaneous Multi-Chip Access and Reconfigurable I/O capability,” CICC, Sept 2003. [7] MPICH – A portable implementation of MPI, http://www.unix.mcs.anl.gov/mpi/mpich/ [8] P. Mosch et al., “A 720 mW 50 MOPS 1V. DSP for a Hearing Aid Chip Set,” Proc. ISSCC, pp. 238-239, Feb. 2000. [9] Õzgün Paker et al., “A heterogeneous multi-core platform for low power signal processing in systems-on-chip,” ESSCIRC 2002. [10] W. Qin, S. Malik, “Flexible and Formal Modeling of Microprocessors with Application to Retargetable Simulation,” Proceedings of DATE 2003, Mar, 2003, pp.556-561. [11] F. Rampogna et al., “Magic, a Low-Power, re-configurable DSP”, Chapter 21 in “Low Power Electronics Design”, ed. C. Piguet, CRC Press, 2004. [12] P. Schaumont, I. Verbauwhede, M. Sarrafzadeh, K. Keutzer, “A quick safari through the reconfiguration jungle,” Proceedings DAC 2001, pg. 172-177, June 2001.
  • 155[13] T. Stefanov, C. Zissulescu, A. Turjan, B. Kienhuis, E. Deprettere,``System Design using Kahn Process Networks: The Compaan/Laura Approach, DATE2004, Feb 2004, Paris, France.[14] T. Stefanov, B. Kienhuis, E. Deprettere, “Algorithmic Transformation Techniques for Efficient Exploration of Alternative Application Instances, Proc. CODES2002, Colorado, May 2002.[15] I. Verbauwhede, J. M. Rabaey. “Synthesis of Real-Time Systems: Solutions and challenges” Journal of VLSI Signal Processing, Vol. 9, No. 1/2, Jan. 1995, pp. 67-88.[16] I. Verbauwhede, M.C. F. Chang, “Reconfigurable Interconnect for next generation systems”, Proc. SLIP, pp. 71-74, April 2002.[17] Xilinx: Virtex-II-Pro Platform FPGAs: Introduction and Overview and Functional Description, Aug. 2003, Oct. 2003, www.xilinx.com/bvdocs/publications/ds083-1.pdf, ds083-2.pdf.[18] H. Zhang, et al., “A 1V Heterogeneous Reconfigurable Processor IC for Baseband Wireless Applications,” IEEE Journal on Solid State Circuits, November 2000.[19] C. Zissulescu, et al., ``Laura: Leiden Architecture Research and Exploration Tool, Proc. FPL 2003.
  • 156Chapter 9SOURCE-LEVEL MODELS FOR SOFTWAREPOWER OPTIMIZATIONCarlo Brandolese, William Fornaciari and Fabio SalicePolitecnico di MilanoAbstract This chapter presents a methodology and a set of models supporting energy-driven source-to-source transformations. The most promising code transformation tech- niques have been identified and studied leading to accurate analytical and/or statistical models. Experimental results obtained for some common embedded- system processors over a set of typical benchmarks are discussed, showing the value of the proposed approach as a support tool for embedded software design.Keywords: Software optimization, Power optimization, Source-level modeling9.1 INTRODUCTION In a growing number of complex heterogeneous embedded systems the rele-vance of the software component is rapidly increasing. Issues such as develop-ment time, flexibility and reusability are, in fact, better addressed by softwarebased solutions. Another trend that is significantly pushing designers to moveas much functionality as possible toward software is the increased interest inplatform-based designs. In such systems much of the architecture is fixed andcan only be configured to match the design constraints. The greatest part ofthe application-specific functionality is thus naturally shifted from hardwarededicated components to software programs. In such a scenario it is clear thatthe importance of software is steadily increasing and poses new problems todesigners. Though performance, in the sense of computational efficiency, isstill the foremost requirement for many embedded systems, power consump-tion is gaining more and more attention. Optimization of the code is thus one ofthe key points and is currently addressed almost only by means of compilationtechniques. It is still not uncommon for designers to manually code criticalsections of the application directly in assembly. The recent technical litera-
  • 157ture proposes a different approach, based on source-to-source transformationsaimed at improving code quality either directly or by enabling better compileroptimizations. Source code transformations are extremely complex to auto-mate since they require a thorough semantic analysis of the code fragmentsto be optimized. This chapter proposes a sound and flexible methodology forthe analysis of the effect of source-to-source transformations mostly aimed atallowing rapid and accurate design space exploration. The proposed approachis based on a wide set of models studied to decouple the processor-independentanalysis from all technology specific aspects.9.2 TRANSFORMATIONS OVERVIEW Source-to-source transformation presented in literature, can be grouped into four main areas according to the code structures they operate on: loops, datastructures, procedures, control structures and operators. It is worth noting thatnot all the transformations are interesting when operating at source-level sincesome of them can as well be performed at RT or assembly-level and are thusperformed by modern compilers. The most promising transformations, eitherfound in literature [1, 2] or studied in the present work, are summarized in thefollowing. Particular attention must be devoted to loop transformations [3–6]since most of the execution time of a program is spent in loops.Loop unrolling replicates the body of a loop a given number of times U (the unrolling factor), and modifies the iteration step from 1 to U . The trans- formation impacts on energy in two ways: on one hand, it reduces loop overhead by performing less compare and branch instructions; on the other hand, it allows the compiler for better optimization and register usage in the larger loop body.Loop distribution breaks a single loop into multiple loops with the same it- eration range but each enclosing only a subset of the statements in the original loop. Distribution is used to create sub-loops with fewer depen- dencies, improve instruction cache and instruction TLB locality due to shorter loop bodies, reduce memory requirements by iterating over fewer arrays and improve register usage by decreasing register pressure.Loop fusion performs the opposite action of distribution, i.e. merging, by re- ducing loop overhead, increasing instruction parallelism, improving reg- ister, data cache, TLB or page locality. It also improves the load balance of parallel loops.Loop interchange exchanges the position of two loops in a loop nest, gener- ally moving one of the outer loops to the innermost position. It is one of the most valuable transformations and can improve performance in
  • 158 many ways: it enables and improves vectorization, increases data access locality and increases the number of loop-invariant expressions in the inner loop.Loop tiling improves memory locality, primarily the at cache level, by access- ing matrices in N ×M sized tiles rather than completely. It also improves processor, register, TLB, and page locality.Software pipelining breaks the operations of a single loop iteration into S stages, and arranges the code in such a way that stage 1 is executed on the instructions originally belonging to iteration i, stage 2 on those of iteration i − 1, etc. Startup code must be generated before the loop to initialize the pipeline for the first S − 1 iterations and cleanup code must be generated after the loop to drain the pipeline for the last S−1 iterations.Loop unswitch is applied when a loop contains a branch with a loop-invariant test condition. The loop is then replicated inside each branch of the con- ditional, saving the overhead of conditional branching inside the loop, reducing the code size of the loop body, and possibly enabling the paral- lelization of one or both branches. The second class collects a number of data-structure and memory accesstransformations [7, 6].Local to global array promotion allows compilers to use simpler addressing modes since global arrays address does not depend on the stack pointer.Scratch-pad array introduction has the goal of storing the most frequently accessed array elements in a smaller array (the scratch-pad) to improve spatial locality.Multiple indirection elimination identifies common chains of indirections and stores the address into a temporary variable. The third group gathers those transformations [7] impacting on proceduresand functions.Function inlining replaces the most frequently invoked function with the func- tion body. Inline expansion increases the spatial locality and decreases the number of function calls. This transformation increases the number of unique references, which may result in more misses. However, a de- crease in the miss rate may also occur, since, without inlining, the callee code might replace the caller code in the instruction cache.Soft inlining is an intermediate solution between function calling and inlining. The transformation replaces calls and returns with jumps. This reduces the code size w.r.t. inlining and eliminates context switching overheads.
  • 159Code linking directives can be used to suitably reorder the objects of different functions to match as more as possible the dynamic call graph. This potentially leads to a reduction in instruction misses. Most of the transformation in the last group are usually performed by com-pilers. Nevertheless, some of them can still be conveniently considered whenoperating at source-level [7, 8].Conditional sub-expression reordering exploits shortcut evaluation of con- ditions usually performed by compilers. The transformation operates by reordering the sub-expressions according to their probability of being true (for OR conditions) or false (for AND conditions). This reduces the number of instructions executed.Special cases pre-evaluation allows avoiding a function call (usually a math- ematic library function) when the argument has a special value for which the result is known. This is done by defining suitable macros testing for the special cases and leads to a reduction of actual calls.Special cases optimization replaces calls to generic library or user-defined functions with optimized versions, suitable for common special cases. As an example, power raising on integers can be coded more efficiently than it can be for real numbers.9.3 METHODOLOGY Transformations applied to source code might lead to very different resultsdepending on a number of factors: the specific structure of the code, the targetarchitecture, the parameters of the transformations etc. Furthermore, it is notunusual that a transformation applied on the source code as it is, leads to pooror no energy reduction, while, when applied to a pre-transformed code itseffectiveness is greatly increased. Thus sequences of transformations shouldbe considered, rather than single transformations. For this reason it is crucialto explore different transformations and sequences of transformations in termsof their energy reduction efficiency. The exploration strategy should allow toeasily modify the parameters of the transformation and of the target technologyand thus leading to a quick estimate of the expected benefits.9.3.1 Conceptual Flow Figure 9.1 shows the conceptual scheme of the estimation flow. The sourcecode is processed and its relevant characteristics are extracted by means of alexical and syntactical analysis leading to the set of code parameters. Typicalparameters are code size, loop body size, number of paths, number of loopiterations, etc.
  • 160 Source Code ↓ Transformation → Code Analysis parameters ↓ ∆I, ∆Minst , ∆Mdata ↓ Technology → Energy Estimation parameters ↓ ∆E Figure 9.1. Phases of the methodology flow The designer then chooses the transformations parameters such as unrollfactor, tiling size etc. and, finally, selects the target technology from a set oflibraries. Such libraries are collections of technology parameters specifyingarchitectural figures such as cache sizes, bus width etc. and electrical figuressuch as power supply voltages, average core currents, bus and memory capaci-tances etc. Based on all this data, the estimation models first provide the threedimensionless figures ∆I, ∆Minst and ∆Mdata expressing the variations ofnumber of instructions executed, of number of instruction cache misses and ofnumber of data cache misses, respectively. These figures, though still ratherabstract, already provide the designer with an indication of the potential bene-fits of a given transformation. To account for the target technology as well, thevariations are fed to a set of models, depending on the technology parameters,leading to an estimate of the energy reduction ∆E deriving from the applicationof the considered transformation.9.3.2 Technology Models Experimental results have shown that the energy consumption of an embed-ded system based on a processor executing some programs can be approximatedby considering three major contributions: the processor core and its on-chipcaches, the system bus and the main memory. All these components can bemodeled at different levels of accuracy by means of equations that involve twosets of parameters: those strictly related to the specific technology and thosesummarizing the properties and the behavior of the code being executed. Inparticular, as outlined above in the description of the conceptual flow, the en-ergy estimates can be based on three execution parameters only: the number ofassembly instructions executed and the number of instruction and data cachemisses. Though simple, the adopted models provide satisfactory results, es-pecially when considering energy variations rather than absolute values. Thetechnology parameters considered and used in the models adopted for the CPU,the cache, the bus and the main memory are summarized in Table 9.1.
  • 161Table 9.1. Technology parametersSymbol Meaning Symbol Meaning Tck CPU clock period B Cache block size CP I Average CPI1 S Cache size P cpu Average CPU power Edec Memory decode energy Ctot Total capacitance on the bus Erw Memory read/write energy Vsw Bus switching voltage Eref Memory refresh energy Asw Average bus switching activity Vm Memory supply voltage W Bus width I ref Average memory refresh current The form of the equations, referred to relative energy variations, are reportedin the following using the symbols introduced. The processor energy variationis modeled as: ∆Ecpu = Tck P cpu CP I∆I (3.1)The contribution of system bus to energy variation ∆Ebus is: ∆Ebus = 1 Ctot Vsw (∆Nbus,addr + ∆Nbus,data + ∆Ninst ) 2 2 (3.2)where: ∆Nbus,addr = Asw,addr Waddr (∆Mdata + ∆Minst ) (3.3) ∆Nbus,data = Asw,data Wdata Bdata ∆Mdata (3.4) ∆Nbus,inst = Asw,instWdata Binst ∆Minst (3.5)Finally, the adopted memory model expresses the energy variation ∆Em as: ∆Em = ∆Em,data + ∆Em,inst + ∆Em,ref (3.6)where: ∆Em,data = (Edec + Erw Bdata )∆Mdata (3.7) ∆Em,inst = (Edec + Erw Binst )∆Minst (3.8) ∆Em,ref = Tck Vm I ref CP I∆I (3.9)9.4 CASE STUDIES In this section, two case studies are reported: Loop unrolling and Loopfusion. For each transformation, the source code parameters and the modelequations are reported and discussed.9.4.1 Loop Unrolling Loop unrolling is a parametric transformation whose results in terms ofenergy reduction are influenced by the unrolling factor U , i.e. the number of
  • 162times the loop body is replicated to build the modified loop. The parameterU , thus, completely defines the transformation. The effects of loop unrollingclearly depend also on the characteristics of the source code being transformed.Such properties are captured by the set of source code parameters reported inTable 9.2.Table 9.2. Source code parameters for loop unrolling Symbol Meaning LI Number of loop instructions LS Size of loop instructions (bytes) LBI Number of loop-body instructions LBS Size of loop-body instructions (bytes) N Loop iterations The number of instructions of the original loop is: Io = N · LI (4.1)The transformed loop executes Nt = N/U times and: LIt = LI + (U − 1)LBI (4.2)instructions per iteration. Therefore, the total number of instructions executedby the transformed loop is: N It = Nt · LIt = · [LI + (U − 1)LBI] (4.3) UThe instructions gain obtained with unrolling is thus: N ∆I = · [LI + (U − 1)LBI] − Io (4.4) UThe transformation has also effects on the number of instruction cache missesdue to the increased dimension of the loop body. A more accurate analysisleads to the results—summarized in the following—that show a non-lineardependence of the number of misses on the relative values of the loop sizeLS and the instruction cache size Sinst 2 . Three significant cases have beenidentified: LS ≤ Sinst In this case there are no capacity misses since the entire loop code can2 The loop size and number of instructions are linearly related assuming a fixed instruction size.
  • 163 be loaded into the cache. Hence, there are only cold misses, during the first iteration. The number of instruction cache misses is thus: LS Minst = (4.5) Binst Sinst < LS < 2Sinst In this case capacity misses also take place. The number of cold misses is the same as in the previous case, but, in addition, for every additional iter- ations, there are 2 (LS mod Sinst )/Binst capacity misses. Therefore, the total number of misses is: LS LS mod Sinst Minst = + 2(N − 1) (4.6) Binst Binst LS ≥ 2Sinst The number of misses in every iteration is equal to the number of cold misses, i.e.: LS Minst = N (4.7) Binst For all these cases, the relevant figure is the variation of the number ofinstruction cache misses ∆Minst = IMt − IMo . Such difference depends onthe variation of number of instructions due to the transformation: ∆LS = LSt − LSo = (U − 1)LBS (4.8)and must be calculated for all the 32 = 9 cases. It is worth noting that sincethe transformed code will always be larger than the original one, only 6 out ofthe 9 cases are significant. For the sake of conciseness, only the two boundarycases are described in the following. (LSo ≤ ICS) ∧ (LSt ≤ ICS) In this case both the original and the transformed code completely fit into the cache and thus only cold misses take place. The variation, recalling Equation (4.5), is: LSo LSt (U − 1)LBS ∆Minst = − ≈ (4.9) Binst Binst Binst (LSo ≥ 2ICS) ∧ (LSt ≥ 2ICS) In this other limiting case, both codes are larger than the double of the cache size and thus each instruction fetch causes a miss. Recalling Equa- tion (4.7), the instruction miss variation is: LS + (U − 1)LBS LS ∆Minst = Nt − No (4.10) IBS IBS
  • 164 In a similar manner and referring to Equations (4.5)–(4.7), the variationsfor the other four cases can be calculated. The last effect to be considered isthe variation of data cache misses. Since the transformation does not modifythe data access pattern of the code, the term ∆Mdata can be assumed to be0, at least at a first approximation. A first validation can be performed atthis level comparing the dimensionless estimated figures ∆I and ∆Minst withthose derived from simulation. Figure 9.2 shows the results for the variation ofnumber of instruction executed. It is worth noting that ∆I does not depend onthe cache size but only on the structure of the code and the effectiveness of theoptimizations that the compiler can exploit on the modified loop. Loop Unrolling -300 Actual Estimated -400 Instruction gain -500 -600 -700 0 10 20 30 40 50 60 70 80 90 100 Unroll factor (U) Figure 9.2. Loop unrolling: ∆I As far as the variation of instruction cache misses, different scenarios havebeen considered by varying the cache size from 256 to 4096 bytes. Table 9.3summarizes the results obtained by averaging the estimation error over theinterval U = [2; 100] and Figure 9.3 shows the two boundary cases.Table 9.3. Loop unrolling: ∆Minst average error and standard deviation Sinst (bytes) % σ% 256 -1.881 8.026 512 -2.557 7.101 1024 -2.531 6.910 2048 -2.750 9.252 4096 -1.691 5.065 The two contributions ∆I and ∆Minst (remembering that ∆Mdata = 0)can now be fed to the technology models to derive the overall energy saving.Table 9.4 reports the average error and the corresponding standard deviation interms of energy gain for the five cache-size scenarios just considered. These results show that the model tends to underestimate the potential gainderiving from loop unrolling. A possible reason is that unrolling a loop leads to a
  • 165 Loop Unrolling 400 Cache size = 256 Bytes Instruction miss gain 300 200 100 Actual Estimated 0 0 10 20 30 40 50 60 70 80 90 100 400 Cache size = 4 KBytes Instruction miss gain 300 200 100 Actual Estimated 0 0 10 20 30 40 50 60 70 80 90 100 Unrolling factor (U) Figure 9.3. Loop unrolling: ∆MinstTable 9.4. Loop unrolling: ∆E average error and standard deviation Sinst (bytes) % σ% 256 -1.754 9.144 512 -4.552 7.322 1024 -7.663 6.966 2048 -6.203 5.777 4096 -4.409 3.011longer loop body, i.e. a larger basic block where the compiler can better performoptimizations. Despite the light biasing of the model, the overall average erroris, in absolute value, approximately 4.9% and this can be considered more thansatisfactory when operating at source code level.9.4.2 Loop Fusion This transformation has the purpose of combining into a new single loopthe bodies of different subsequent loops. Some constraint must be satisfied,in particular the loops to be merged need to have the same iteration range andthe statements in their bodies must be independent. The only transformationparameter characterizing loop fusion is the number N F of loops to be merged.The source code parameters that influence the effect of this transformation are
  • 166all those considered for loop unrolling (see Table 9.2) plus the number and sizeof control instructions, defined as: LCI = LI − LBI (4.11) LCS = LS − LBS (4.12) In the following the subscript k ∈ [1, N F ] is used to indicate a specific loopamong those to be fused. An additional useful parameter is the average numberof control instructions over all the considered loops: NF 1 LCI = LCIk (4.13) NF k=1 Using the symbols just introduced, the number of instructions in the originaland transformed codes are: NF Io = N (LBIk + LCIk ) (4.14) k=1 NF It = N (LCI + LBIk ) (4.15) k=1 The variation ∆I is thus given by: NF NF ∆I = N (LCI + LBIk − (LBIk + LCIk ) = k=1 k=1 (4.16) NF = N (LCI − LCIk ) k=1 Assuming that LCI = LCIk ∀k yields: NF NF LCIk = LCI = N F · LCI (4.17) k=1 k=1 and thus Equation (4.16) can be rewritten as: NF ∆I = N (LCI − LCIk ) = N (1 − N F )LCI (4.18) k=1
  • 167 To study the effect of loop fusion with respect to instruction misses, thesame cases considered for loop unrolling and expressed by Equations (4.5)–(4.7) turn out to be applicable. Nevertheless, when considering the original codecomposed of N F loops, the number of instruction misses must be estimated foreach single loop according to the three mentioned equations and then summedover all loops. On the other hand, the estimates for the transformed code canbe obtained by simply substituting LS with the overall transformed code sizeLSt , defined as: NF LSt = LCS + LBSk (4.19) k=1 According to Equations (4.5)–(4.7) and referring to the original code sizesLSo,k and the transformed code size LSt , the number of instruction misses ofthe original loops IMo,k and the transformed one IMt can be derived. Theresulting overall variation is thus: NF ∆Minst = IMt − IMo,k (4.20) k=1 It is worth noting that the number of possible cases derived from the limitingconditions on the cache size is, in general, 3N F +1 . Similar considerationsapply to the estimation of data cache misses. Since in most cases the differentloops operate on different arrays, data misses tend to be increased, the best-case condition being that all data fit into the cache in which case the number ofmisses will approximately be invariant. A validation procedure similar to thatused for loop unrolling has been applied for loop fusion also, considering thesimplest and most common case where N F = 2. To analyze the behavior ofthe transformation, loops with different body sizes have been considered andthe results for instruction misses are shown in Figure 9.5, where the x axis isan index related to the loop body size ratio. For the same combinations of loopbody sizes and for an instruction cache size varying from 256 to 4096 bytes,the gain in terms of instruction misses have also been estimated and comparedwith actual results, leading to the data collected in Table 9.5 and the graphs ofFigure 9.5 relative to the two limiting cases. Again the accuracy obtained is more than satisfactory since the average abso-lute error is approximately 2.1% with very low standard deviation. Combiningdimensionless figures with the energy models of the different component ofthe considered system led to the energy estimates. Such estimates show a verylimited error, as reported in Table 9.6, and are not biased. It is though worthnoting that the reported results refer to loops manipulating very small arraysfor which the hypothesis of being fully contained in the data cache may beassumed to hold. This translates into the models by assuming ∆Minst = 0.
  • 168 Loop Fusion -600 Actual Instruction gain -700 Estimated -800 -900 -1000 -1100 -1200 0 10 20 30 40 50 60 Loop size index Figure 9.4. Loop fusion: ∆ITable 9.5. Loop fusion: ∆Minst Sinst (bytes) % σ% 256 +2.423 2.701 512 +3.004 2.804 1024 -3.150 4.253 2048 +0.153 1.672 4096 -0.258 1.419 Loop Fusion 1500 Cache size = 64 Bytes Instruction miss gain 1000 Actual Estimated 500 0 -500 0 10 20 30 40 50 60 0 Cache size = 4 KBytes Instruction miss gain -1 -2 -3 -4 Actual Estimated -5 0 10 20 30 40 50 60 Loop size index Figure 9.5. Loop fusion: ∆MinstMore complex cases show higher errors but preliminary experimental resultssuggest that a 10–15% error is a reasonable and conservative upper bound.
  • 169Table 9.6. Loop fusion: ∆E Sinst (bytes) % σ% 256 +1.945 3.882 512 +0.177 3.469 1024 -0.194 3.916 2048 +1.592 2.425 4096 +0.168 0.0179.5 EXPERIMENTAL RESULTS The estimates of ∆I, ∆Minst and ∆Mdata , combined with the energy models(see Section 9.3.2) adopted to account for the technology-dependent parameters,lead to a new set of results showing the accuracy of the complete methodology interms of energy reduction (∆E) estimation. The models for 5 transformationshave been tested on a set of SPEC95 benchmarks in order to quantify theenergy gain estimation error. The actual energy gain has been obtained bysimulating both the original and the transformed code and then compared withthe estimated gain derived from the models. Experiments have been performedon four architectures based on different processors and operating systems usingthird-party timing and/or power profiling tools (see Table 9.7).Table 9.7. Operating environments for validation Processor Operating system Simulation engine Intel strongARM Linux RedHat 9.0 SimpleScalar 3.0 / SimPAnalyzer IBM PowerPC 405 Linux RedHat 9.0 SimpleScalar 3.0 Sun microSPARC II EP Solaris 8 SpixTools MIPS Tech. MIPS-32 Linux RedHat 9.0 SimpleScalar 3.0 Each benchmark has been analyzed varying both the instruction cache size(Sinst ) and the input data and all compatible transformations have been appliedin a proper sequence using the predicted optimal values for their parameters(unroll factor, tile size, etc.). Table 9.8 collects the relative error between theestimated gain ∆Eest and the actual value ∆Eact derived from simulation. The results confirm that the models are reliable since they can correctlypredict both energy reductions and undesirable energy increases. In conclusion,the average estimation error has shown to be around than 3%.
  • 170Table 9.8. Energy gain estimation relative errors FIB FIR WAVE-1 WAVE-2 IIR Sinst % σ% % σ% % σ% % σ% % σ% 256 +4.16 3.90 n/a n/a -1.97 2.81 +4.29 3.63 -1.63 1.20 512 +7.18 4.02 -3.67 4.48 -1.83 2.67 +4.63 3.52 -1.82 1.15 1024 +3.31 1.49 -2.11 4.95 -2.87 3.51 +4.81 0.79 -3.93 1.51 2048 -1.42 2.15 +1.03 7.68 -2.37 3.71 +4.20 0.57 -0.53 1.59 4096 -2.08 1.91 +11.25 7.57 -1.86 3.71 +3.74 0.20 +0.03 16.00Average 3.63 2.69 4.51 6.17 2.18 3.28 4.33 1.74 1.58 4.299.6 CONCLUSIONS The presented work has addressed the problem of the fast estimation of theeffects induced by a set of specific source code transformations by using astructured methodological approach based on technology-independent models.In particular, the presented analysis flow, by providing an appropriated set ofboth technological and transformation parameters, allows the designer to an apriori evaluation of the impact of a specific transformation and/or the effectof a sequence of interdependent transformations. Two specific transformationshave been accurately described: loop unrolling and loop fusion. As far as loopunrolling is concerned, it has been shown that the proposed model can be con-sidered more than satisfactory since the average error between the estimatedgain and the simulated gain is, approximately, 4,9% with a low standard devi-ation. Concerning loop fusion, the model has produced estimates—for a wideset of technological options—displaying an average absolute error of 2,1% withan high level of reliability. Both the methodology and the models has been val-idated on a set of benchmarks showing an overall average error of the estimatedenergy gain around 3%. This result is more than satisfactory and confirmsthat the models of the different transformation are sufficiently accurate and themethodology, though subject to further improvements, is promising.
  • 171References[1] L. Benini and G. De Micheli. System-level power optimization: Tech- niques and tools. Transactions on Design Automation of Electronic Sys- tems, 5:115–192, 2000.[2] F. Catthoor, H. De Man, and C. Hulkarni. Code transformations for low power caching in embedded multimedia processors. Proc. of IPPS/SPDP, pages 292–297, 1998.[3] D.F. Bacon, S.L. Graham, and O.J. Sharp. Compiler transformations for high performance computing. Technical Report N. UCB/CSD-93-781, Uni- versity of California at Berkeley, 1993.[4] M.S. Lam. Software pipelining: An effective scheduling technique for vliw machines. SIGPLAN Conference on Programming Language Design and Implementation, pages 318–328, 1988.[5] M.S. Lam, E.E. Rothberg, and M.E. Wolfe. The cache performance and optimization of blocked algorithms. Conference on Architectural Support for Programming Languages an Operating Systems, pages 63–74, 1991.[6] M.J. Wolfe. More iteration space tiling. ACM Proceedings of Supercom- puting, pages 655–664, 1989.[7] C. Brandolese, W. Fornaciari, F. Salice, and D. Sciuto. The impact of source code transformations on software power and energy consumption. Journal of Circuits, Systems and Computers, 11(5):477–502, 2002.[8] C. Brandolese, W. Fornaciari, F. Salice, and D. Sciuto. Library functions timing characterization for source-level analysis. Conference on Design Automation and Testing in Europe, pages 1132–1133, March 2003.
  • 172 Chapter 10 TRANSMITTANCE SCALING FOR REDUCING POWER DISSIPATION OF A BACKLIT TFT-LCD Wei-Chung Cheng and Massoud Pedram University of Southern California Abstract This chapter presents transmittance scaling; a technique aimed at conserving power in a transmissive TFT-LCD with a cold cathode fluorescent lamp (CCFL) backlight by reducing the backlight illumination while compensating for the luminance loss. This goal is accomplished by adjusting the transmittance function of the TFT-LCD panel while meeting an upper bound on a contrast distortion metric. Experimental results show that an average of 3.7X power saving can be achieved for still images with a mere 10% contrast distortion. Keywords: CCFL; transmissive LCD; TFT-LCD; backlight luminance dimming; transmittance scaling; concurrent brightness and contrast scaling; power efficiency; low power design. 10.1 INTRODUCTION TFT-LCD is the most popular flat-panel display used in todays consumer electronics and computer systems. TFT stands for "Thin Film Transistor" and describes the control elements that actively control the individual pixels. For this reason, one speaks of so-called "active matrix TFTs". LCD means "Liquid Crystal Display" and stands for monitors that are based on liquid crystals. To obtain a high image quality and low power dissipation in a TFT- LCD, low off-current and high on-current are necessary. Previous studies on battery-powered electronics point out that the display subsystem dominates the energy consumption of the whole system. In the SmartBadge system, for instance, the display consumes 29%, 29%, and 50% of the total power in the active, idle, and standby modes, respectively [1]. Direct-view LCDs can largely be categorized into reflective and transmissive displays which utilize ambient light and light from an artificial light source
  • 173(e.g., fluorescent backlight tube) respectively. In a transmissive TFT-LCDmonitor, the backlight contributes more than 50% of the display subsystemwhen using a cold cathode fluorescent lamp (CCFL) [2]. To reduce thebacklight power consumption, Choi et al. proposed a technique calledbacklight luminance dimming. This technique dims the backlight andcompensates for the luminance loss by adjusting the grayscale of the imageto increase its brightness or contrast. The grayscale of the image is adjustedby multiplying the pixel values by a scaling factor. In this chapter, wedescribe the transmittance scaling technique, which compensates for theluminance loss by adjusting the transmittance function of the TFT-LCDpanel. More precisely, transmittance scaling means “scaling thetransmittance function of the TFT-LCD panel.” This is a general techniquethat can achieve concurrent brightness and contrast scaling of the wholeimage to compensate for the effects of the backlight dimming. In the following sections, we explain how CCFL works and show how tomodel the non-linearity between its backlight illumination and powerconsumption. Next, we propose a contrast distortion metric to quantify theimage quality loss after transmittance scaling. Finally, we formulate andoptimally solve the optimal transmittance scaling problem subject to aconstraint on the contrast distortion.10.2 PRELIMINARIES A transmissive LCD uses a dedicated backlight. A reflective LCD uses theambient light or/and a dedicated frontlight. A transflective LCD uses boththe ambient light and backlight. The frontlight and backlight use the samelight source. The difference between the two lighting schemes is in the lightpath from the light source through the LCD panel to the observer. A back-litor front-lit LCD offers superior contrast ratio compared to the one that is litby the ambient light. A backlight can be direct or indirect type. A directbacklight is positioned directly beneath the LCD panel. An indirect (or side-lit) backlight is positioned at the side of the LCD panel and requires acarefully designed light-guide and a diffuser to illuminate the LCD panelevenly. Most TFT-LCD monitors use CCFL for backlighting due to its unrivaledluminance density – emitting the most light within the minimum form factor.The CCFL can be designed to generate an arbitrary color, which is criticalfor reproducing pure white in the backlighting applications. Technology forCCFL manufacturing is mature; therefore, its production cost is rather low.However, compared to power consumption of the TFT-LCD panel, thepower consumption of the CCFL backlight is quite high.
  • 174 10.2.1 Radiometry and Photometry Terminology Radiometry refers to the science of measuring light in any portion of the electromagnetic spectrum [3]. In practice, radiometry is usually limited to the measurement of infrared, visible, and ultraviolet light using optical instruments. Light is radiant energy. Electromagnetic radiation transports energy through space. Radiant energy (denoted as Q) is measured in joules. A broadband source such as the Sun emits electromagnetic radiation throughout most of the electromagnetic spectrum, from radio waves to gamma rays. However, most of its radiant energy is concentrated within the visible portion of the spectrum. A single-wavelength laser, on the other hand, is a monochromatic source; all of its radiant energy is emitted at one specific wavelength. Energy per unit time is power, which we measure in joules per second, or watts. A laser beam, for example, has so many watts of radiant power. Light “flows” through space, and so radiant power is more commonly referred to as the “time rate of flow of radiant energy” or radiant flux. It is defined as: =dQ/dt where Q is radiant energy and t is time. Radiant flux is measured in watts. In terms of a photographic light meter measuring visible light, the instantaneous magnitude of the electric current is directly proportional to the radiant flux. The total amount of current measured over a period of time is directly proportional to the radiant energy absorbed by the light meter during that time. Radiant flux density is the radiant flux per unit area at a point on a surface. There are two possible conditions. The flux can be arriving at the surface, in which case the radiant flux density is referred to as irradiance. The flux can also be leaving the surface due to emission and/or reflection. The radiant flux density is then referred to as radiant exitance. Radiant flux density is measured in watts per square meter. The radiant flux density at a point on a surface due to a single ray of light arriving (or leaving) at a solid angle to the surface normal is d /(dA·cos ). The radiance at that point for the same angle is then d2 /(dA·d ·cos ), or radiant flux density per unit solid angle. Radiance is measured in watts per square meter per steradian. We can imagine an infinitesimally small point source of light that emits radiant flux in every direction. The amount of radiant flux emitted in a given direction can be represented by a ray of light contained in an elemental cone. This gives us the definition of radiant intensity: I =d /d . Radiant intensity is measured in watts per steradian. Photometry is the science of measuring visible light in units that are weighted according to the sensitivity of the human eye [3]. It is a quantitative science based on a statistical model of the human visual response to light -- that is, our perception of light -- under carefully
  • 175controlled conditions. The human visual system is a complex and highlynonlinear detector of electromagnetic radiation with wavelengths rangingfrom 380 to 770 nanometers (nm). The sensitivity of the human eye to lightvaries with wavelength. A light source with a radiance of one watt/m2-steradian of green light (540nm wavelength), for example, appears muchbrighter than the same source with a radiance of one watt/m2-steradian of red(650nm wavelength) or blue light (450nm wavelength). In photometry, weattempt to measure the subjective impression produced by stimulating thehuman eye-brain visual system with radiant energy. This task is complicatedimmensely by the eye’s nonlinear response to light. It varies not only withwavelength but also with the amount of radiant flux, whether the light isconstant or flickering, the adaptation of the iris and retina, the spatialcomplexity of the scene being perceived, the psychological andphysiological state of the observer, and a host of other variables [4]. According to studies done by the Commission Internationale d’Eclairage(CIE), the photopic luminous efficiency of the human visual system as afunction of wavelength looks like a near-normal distribution as depicted inFigure 10-1 (cf. [5].) The CIE photometric curve thus provides a weightingfunction that can be used to convert radiometric measurements intophotometric measurements. Today the international standard for a lightsource is a point source that has a luminous intensity of one candela (theLatin word for “candle”). It emits monochromatic radiation with a frequencyof 540*1012 Hertz (or approximately 555nm, corresponding with thewavelength of maximum photopic luminous efficiency) and has a radiantintensity (in the direction of measurement) of 1/683 watts per steradian. Figure 10-1: Photopic luminosity function. Together with the CIE photometric curve, candela provides the weightingfactor needed to convert between radiometric and photometricmeasurements. Consider, for example, a monochromatic point source with a
  • 176 wavelength of 510nm and a radiant intensity of 1/683 watts per steradian. The photopic luminous efficiency at 510nm is 0.503. The source therefore has a luminous intensity of 0.503 candela. Luminous flux is the photometrically-weighted radiant flux (power). Its unit of measurement is the lumen, defined as 1/683 watts of radiant power at a frequency of 540*1012 Hertz. As with luminous intensity, the luminous flux of light with other wavelengths can be calculated using the CIE photometric curve. Luminous energy is photometrically-weighted radiant energy. It is measured in lumen seconds. Luminous flux density is photometrically-weighted radiant flux density. Luminous flux density is measured in lumens per square meter. Illuminance is the photometric equivalent of irradiance, whereas luminous exitance is the photometric equivalent of radiant existence. Illuminance can be used to characterize the luminous flux emitted from a surface. Most photographic light meters measure the illuminance. Luminance is photometrically-weighted radiance. In terms of visual perception, we perceive luminance. It is an approximate measure of how “bright” a surface appears when we view it from a given direction. Luminance is measured in lumens per square meter per steradian. The maximum brightness of a CRT or LCD monitor is described by luminance in its specification. Luminous intensity is photometrically-weighted radiant intensity. It is measured in lumens per steradian (i.e., candelas). Luminous intensity can be used to characterize the optical power emitted from a spot light source, such as a light bulb. There is much more that we have not covered here, such as reflectance, transmittance, absorption, scattering, diffraction, and polarization. We have also ignored the interaction of the human visual system with light, including scoptic and mesopic luminous efficiency, temporal effects such as flicker, and most important, color perception. The study of light and how we perceive it fills volumes of research papers and textbooks. 10.2.2 Cold Cathode Fluorescent Lamp A transmissive LCD uses a dedicated backlight. A reflective LCD uses the ambient light or/and a dedicated frontlight. A transflective LCD uses both the ambient light and backlight. The frontlight and backlight use the same light source. The difference between the two lighting schemes is in the light path from the light source through the LCD panel to the observer. A back-lit or front-lit LCD offers superior contrast ratio compared to the one that is lit by the ambient light. A backlight can be direct or indirect type. A direct backlight is positioned directly beneath the LCD panel. An indirect (or side- lit) backlight is positioned at the side of the LCD panel and requires a carefully designed light-guide and a diffuser to illuminate the LCD panel
  • 177evenly. Most TFT-LCD monitors use CCFL for backlighting due to its unrivaledluminance density – emitting the most light within the minimum form factor.The CCFL can be designed to generate an arbitrary color, which is criticalfor reproducing pure white in the backlighting applications. Technology forCCFL manufacturing is mature; therefore, its production cost is rather low.However, compared to power consumption of the TFT-LCD panel, thepower consumption of the CCFL backlight is quite high. A CCFL backlight module consists of the fluorescent lamp, the drivingDC-AC inverter, and the light reflector. A CCFL is a sealed glass tube withelectrodes on both ends. The tube is filled with an inert gas (argon) andmercury. The inner glass surface of the tube is coated with phosphor, whichemits visible light when excited by photons. The wavelength of the visiblelight (i.e., color) depends on the type of the gas and phosphor. In the LCDbacklighting application, a proper mix of red, green, and blue phosphorsproduces the desired three-band white light. In other applications where thepure white is not required, the emitted light is color-shifted (e.g., the bluishor yellowish household hot cathode fluorescent lamps). The CCFL converts electrical energy into visible light by a process knownas the gas discharge phenomenon. When a high voltage is applied to theelectrodes turning on the lamp, electrical arcs are generated that ionize thegas, which allows the electrical current to flow. The collision among themoving ions injects energy to the mercury atoms. The electrons of themercury atoms receive energy and jump to a higher energy level followed byemitting ultraviolet photons when falling back to their original energy level.The ionized gas conducts the electrical current as a gas conductor. Theimpedance of the gas conductor, unlike that of the metal conductor having alinear behavior, decreases as the current increases. Therefore, the CCFL hasto be driven by an alternative current (AC). Otherwise, the CCFL explodesdue to the heat caused by the runaway current. The CCFL is usuallymanufactured as a thin long tube, which can be straight or in any differentshape. A DC-AC inverter is usually used to drive a CCFL in battery-poweredapplications. A DC-AC inverter is basically a switching oscillator circuit thatsupplies high-voltage AC current from a low-voltage battery. The nominalAC frequency of modern CCFL is in the range of 50-100 kHz to avoidflickering. The nominal operate voltage has to be higher than 500 VRMS tokeep inert gas ionized. To conserve energy in battery-powered applications, dimming control is adesirable feature for DC-AC inverters. Different methods of dimming theCCFL have been used, including linear current, pulse-width-modulation, andcurrent chopping [6]. In a DC-AC inverter with dimming control, an analog
  • 178 or digital input signal is exposed for adjusting the CCFL illumination in addition to the on/off input signal. Most well designed DC-AC inverters have high electrical efficiency (>80%) and linear response of output electrical power to input power. Most fluorescent lamps, however, have low optical efficiency (<20%) and non-linear response of output optical power versus input power [7]. 10.2.3 Power Modeling for the CCFL The CCFL illumination is a complex function of the driving current, ambient temperature, warm-up time, lamp age, driving waveform, lamp dimensions, and reflector design [7]. In the transmissive TFT-LCD application, only the driving current is controllable. Therefore, we model the CCFL illumination as a function of the driving current only and ignore the other parameters. 1 0.9 CCFL illuminance 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.5 1 1.5 2 2.5 3 Pow er Consum ption (W) Figure 10-2: Normalized CCFL illuminance (i.e., backlight factor, b) versus driver’s power consumption for a CCFL light source. Relationship between the CCFL illumination (i.e., luminous flux incident on a surface per unit area) and the driver’s power dissipation for the CCFL in LG Philips transmissive TFT-LCD LP064V1 [8] is shown in Figure 10-2. The CCFL illumination increases monotonically as the driving power increases from 0 to 80% of the full driving power. For values of driving power higher than this threshold, the CCFL illumination starts to saturate. The saturation phenomenon is due to the fact that the increased temperature and pressure inside the tube adversely impact the efficiency of emitting visible light [9].
  • 179 Let backlight factor b∈[0,1] denote the normalized CCFL illuminationfactor with b=1 representing the maximum backlight illumination and b=0representing no backlight illumination. Accounting for the saturationphenomenon in the CCFL light source, we use a two-piece linear function tocharacterize the power consumption of CCFL as a function of the backlightfactor: Alin .b +Clin , 0≤b≤ Bs Pbacklight (b ) = (Watt). (1) Asat .b +Csat , Bs ≤b≤1 We conduct experiments to derive the regression coefficients, A’s and C’s.A precision luminance meter such as [10] provides accurate absoluteilluminance readings. These meters are expensive and can be replaced withthe cheaper photographic light meters. Indeed the absolute illuminancereadings are not required to characterize the CCFL power consumption as afunction of the backlight factor. An accurate photographic light meter canserve the purpose so far as it is capable of sensing minor illuminancechanges as described next. We use a light meter in a way similar to how we compare the mass of twoobjects on a scale (weighing machine). We simultaneously adjust thebacklight factor b and the TFT-LCD grayscale, x∈[0,255], whilemaintaining the same CCFL illuminance. Let q(b) denote the required analogor digital dimming control input of the DC-AC inverter for generatingbacklight factor b. We start by setting the maximum CCFL illuminance(b=1) and the minimum TFT-LCD grayscale (x=0). The grayscale x isobtained by displaying a pure gray image in which Red=Green=Blue=x forevery pixel. The grayscale x is increased until the light meter senses avariation and reports a different reading. We then reduce the backlight factorb (by adjusting the dimming control q) until the meter reports the previousreading. The change in the TFT-LCD grayscale (which determines thetransmittance of the TFT-LCD panel) is thus known. Now the change in thebacklight factor is assumed to be the equal to the TFT-LCD transmittance.We record q as the dimming control value for the backlight factor b=(255-x)/256. At the same time, the power consumption of the backlight Pbacklight ismeasured and recorded. We repeat the above procedure for x=0,1...,255.After interpolation, we obtain q(b) and Pbacklight(b). For the CCFL in LGPhilips transmissive TFT-LCD LP064V1, the following coefficient valueswere obtained: Alin=0.4991, Asat=0.1489, Clin=0.1113, Csat=0.6119, (2) Bs=0.8666.
  • 180 10.2.4 Transmissive TFT-LCD Monitor The major components of a transmissive TFT-LCD monitor subsystem include a video controller, a frame buffer, a video interface, a TFT-LCD panel, and the backlight (cfr. Figure 10-3.) The frame buffer is a portion of memory used by software applications to deliver video data to the video controller. The video data from the application is stored in the frame buffer by the CPU. The video controller fetches the video data and generates appropriate analog (e.g., VGA) or digital (e.g., DVI) video signals to the video interface. The video interface carries the video signals between the video controller and the TFT-LCD monitor. The TFT-LCD monitor receives the video data and generates a proper shade – i.e., transmittance – for each pixel according to the corresponding pixel value. All of the pixels on a transmissive LCD panel are illuminated from behind by the backlight. To the observer, a displayed pixel looks bright if its transmittance is high (i.e., it is in the on state), meaning it passes the backlight. On the other hand, a displayed pixel looks dark if its transmittance is low (i.e., it is in the off state), meaning that it blocks the backlight. If the transmittance can be adjusted to more than two levels between the on and off states, then the pixels can be displayed in grayscale. If the shade can be colored as red, green, or blue by using different color filters, then pixels can be displayed in color by mixing three sub-pixels each in a different color and with its own grayscale. In other words, the perceived brightness of a pixel is determined by its transmittance and the backlight illumination. Grayscale Control Display CPU Line Video data Scanner Backlight Frame TFT-LCD Video Timing panel Power supply DC-AC Figure 10-3: Block diagram of a TFT-LCD monitor.
  • 181 A TFT-LCD panel consists of the following ordered layers: frontpolarizer, color filter, glass, indium tin oxide (ITO), polymide film, liquidcrystals, polymide film, ITO, glass, and rear polarizer. The lighttransmittance is determined by the front and rear polarizers and theorientation of the liquid crystals. A polarizer is a light filter that blocks thelight wave in different directions. After passing thought the rear polarizer,the backlight becomes in a single direction. If the front polarizer and rearpolarizer are perpendicular to one another and there is no liquid crystal inbetween, then the backlight will be blocked and the LCD looks dark.Otherwise, the backlight will pass through and the LCD looks bright. Theliquid crystals can be considered as tiny lenses between the two polarizers.The direction of light wave can be twisted by changing the orientation of theliquid crystals. Thus, the liquid crystals can be considered as voltage-controlled light switches (cfr. Figure 10-4.) Figure 10-4: LCD as voltage-controlled light switch. The liquid crystals are in a phase of matter between liquid and solid states.In the liquid state the molecules can move freely, whereas in solid state themolecules are fixed in certain order. The liquid crystals are in the state thatthe molecules can move until they form a certain order according to anexternal force. Three types of liquid crystal phases are used in LCD:nematic, cholesteric, and smectic. The twisted nematic (TN) liquid crystal isthe most widely used in todays LCD monitors.
  • 182 The nematic liquid crystals can be considered as transparent rods (rod-like lenses.) These rods are locally aligned with their long axes nearly parallel to each other on a two-dimensional plane. Their orientation can be denoted by an angle. When there are two stacked planes of nematic liquid crystals, the molecules on each plane align with each other, but the angle of each plane can be different, or twisted. When more twisted planes of nematic liquid crystals are stacked, a series of twisted rod-like lenses forms a spiral light path, which can twist the direction of the light wave from the backlight. If the polarizers are perpendicular, then the backlight passes and a bright pixel is seen. This configuration of crossed polarizers is called normally white. When an external electrical field is applied, all nematic liquid crystals change their orientation such that their long axes point to the electrodes uniformly. In this case, the direction of the light wave is not affected and forms a dark pixel in a normally white LCD. In other words, a normally white LCD consumes power to generate a dark pixel. By controlling the electrode voltage, the amount of light passing the LCD can be modulated to generate grayscale pixels. The nematic liquid crystals are sandwiched between two glasses. The spacing in between decides the number of planes of nematic liquid crystals. To twist the liquid crystals, parallel groves are produced on the glasses by using polymide film. The groves on the front glass set the orientation of the nearby nematic liquid crystals. The groves on the rear glass set the orientation of the nearby nematic liquid crystals to be perpendicular to the first one. The orientations of other planes in between are twisted by the first and last plane from 0 to 90 degree accordingly when no voltage applied. The color filter determines a sub-pixel to be red, green, or blue. Vg Vd Vcom Figure 10-5: The electrical waveforms of the gate-source Vg and drain-source voltage Vd of a TFT-LCD. Each sub-pixel has an individual liquid crystal cell, a thin-film-transistor and a storage capacitor. The layout of the transparent ITO electrodes defines the shape of a sub-pixel. The liquid crystals between the electrodes form a
  • 183conceptual cell. The electrical field of the capacitor controls thetransmittance of the cell. The capacitor is charged and discharged by its ownTFT. The gate electrode of the TFT controls the timing ofcharging/discharging when the pixel is scanned (or addressed) for refreshingits content. The (drain-) source electrode of the TFT controls the amount ofcharge (cfr. Figure 10-5). Vcom Vd Vg Gate bus line G Cgs Vn D S Cst Clc Vs Vcom t Source bus line Figure 10-6: The equivalent circuit of a TFT-LCD sub-pixel. CST is the storage capacitor. CGS and Clc are parasitic capacitances.The gate electrodes and source electrodes of all TFT’s are driven by a set ofgate drivers and source drivers, respectively. A single gate driver (called agate bus line) drives all gate electrodes of the pixels on the same row. Thegate electrodes are enabled at the same time the row is scanned. A singlesource driver (called a source bus line) drives all source electrodes of thepixels on the same column (cfr. Figure 10-6). The source driver supplies thedesired voltage level (called grayscale voltage) according to the pixel value.In other words, ideally, the pixel value transmittance, t(x), is a linearfunction of the grayscale voltage v(x), which is itself a linear function of thepixel value x. If there are 256 grayscales, then the source driver must be ableto supply 256 different grayscale voltage levels. For the source driver toprovide a wide range of grayscales, a number of reference voltages arerequired. The source driver mixes different reference voltages to obtain thedesired grayscale voltages. Typically, these different reference voltages arefixed and designed as a voltage divider. For example in [8], an AnalogDevices input LCD reference driver [11] is used with a 10-way voltagedivider.
  • 184 Vdd rk Vk-1 rk-1 Vk-2 rk-2 Vk-3 rk-3 V2 r2 V1 r1 V0 r0 Figure 10-7: The voltage divider generating reference voltages for grayscale controller. Assume that the transmittance of the TFT-LCD is linear and the resistors of the voltage divider are identical. If k+1 identical resistors r0…rk are connected in series between ground and Vdd in that order, then the output voltage seen from the top terminal of ri is (cfr. Figure 10-7): i +1 Vi = V . (3) k + 1 dd 1 0.9 Transmittance t 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.96 0.965 0.97 0.975 0.98 0.985 0.99 0.995 1 TFT-LCD Panel Power Figure10- 8: Normalized pixel transmittance, t(x), versus power consumption of a pixel in the LCD panel.
  • 18510.2.5 Power Modeling for the TFT-LCD The hydrogenated amorphous silicon (a-Si:H) is commonly used tofabricate the TFT in display applications. For a TFT-LCD panel, the a-Si:HTFT power consumption can be modeled by a quadratic function of pixelvalue x∈[0,255] [13]: PTFT (x)=c0+c1x+c2x2 (Watt). (4) We performed the current and power measurements on the LG Philips,LP064V1 LCD. The measurement data are shown in Figure10- 8. Theregression coefficients are thus determined as: c0=2.703E-3, c1=2.821E-4, c2=2.807E-5. (5) The power consumption of a normally white TFT-LCD panel decreases asits global transmittance increases. In other words, while maintaining thesame luminance, the power consumption of the TFT-LCD decreases whendimming the backlight. The change in the TFT-LCD power consumption is,however, quite small.10.3 BACKLIGHT AND TRANSMITTANCE SCALING The general framework for the backlight luminance dimming and thetransmittance scaling is depicted in Figure 10-9. The pixel illumination isdetermined by the backlight illumination and the LCD transmittance. Thebacklight illumination is controlled by adjusting the amplitude of thedimming control signal to the DC-AC inverter. The LCD transmittance iscontrolled by the pixel values and the grayscale controller. The backlightscaling technique dims the backlight to save power and increases pixelvalues to compensate for the brightness loss. The transmittance scalingtechnique dims the backlight to save power and increases the referencegrayscale voltages to compensate for the brightness loss. Note thattransmittance scaling does not change the pixel values.
  • 186 Dimming value Pixel value Reference 10 Control voltage Grayscale controller grayscale voltages DC-AC inverter Source driver CCFL Illumination * LCD Transmittance = Pixel Illumination Figure 10-9: A framework for backlight luminance dimming and transmittance scaling. The luminance of a transmissive object is the product of the luminance of the incident light and the transmittance of the object [5]. For a pixel on a transmissive TFT-LCD monitor, its transmittance, t(x), is a function of its pixel value x. More precisely, a pixel value of zero means zero transmittance and hence the perceived shade will be black whereas a pixel value of 255 means a transmittance of one and hence the perceived shade will be white. Other pixel values between 0 and 255, result in various shades of gray. Now, luminance, L(x), of a pixel in the TFT-LCD panel is calculated as: L(x) = b·t(x) (6) The ambient light is not considered here because it has little effect for a transmissive TFT-LCD when compared with a reflective or transflective one. Figure 10-10 depicts the relation in (6) assuming that the TFT-LCD transmissivity is a linear function of the pixel value. b * = Backlight TFT-LCD Transmittance Luminance Function Factor Function Figure 10-10: The luminance of a normalized pixel value (right) is the product of the backlight factor b and the TFT-LCD transmissivity function. In a non-backlight-scaled TFT-LCD monitor, the backlight luminance (denoted by the normalized backlight factor b) is fixed at full CCFL driver power (b=1).
  • 18710.3.1 Backlight Luminance Dimming Reference [2] describes two backlight luminance dimming techniques,which dim the backlight luminance to save power consumption. Tocompensate for brightness loss, the authors reduce b while increasing thepixel values from x to x by two mechanisms. The “backlight luminancedimming with brightness compensation” technique uses: x’=x+b. (7) The “backlight luminance dimming with contrast enhancement” techniqueuses: x’=x/b. (8)The pixel values are adjusted by software before being written into the framebuffer or by hardware after being fetched by the video controller. Thedistortion after backlight luminance dimming is evaluated by the percentageof saturated pixels that exceed the range of pixel values, i.e., [0,255]. Theoptimal backlight factor is determined by the backlight luminance dimmingpolicy subject to the given distortion rate. To calculate the distortion rate, ahistogram estimator is required for calculating the statistics of the inputimage.10.3.2 Programmable LCD Reference Driver Recall that the pixel value transmittance, t(x), is a linear function of thegrayscale voltage v(x). The transmittance scaling approach is to control themapping of v(x) in order to control the transmittance function t(x). Wepropose using a programmable LCD reference driver (PLRD) described asfollows. The PLRD is implemented by adding an extra logic to the original voltagedivider expressed by (6). The logic contains a number of p-channel and n-channel switches and multiplexers. Recall that k+1 identical resistors r0…rkare connected in series between ground and Vdd. The PLRD takes two inputarguments gl and gu, and then short circuits the top terminal of rgl to groundand the top terminal of rgu to Vdd. In this way, the output voltage seen fromthe top terminal of ri becomes: Vdd , gu ≤i ≤ k i − gl Vi , gl , gu = Vdd , gl ≤i < gu . (9) gu − gl 0, 0≤i ≤ gl Clearly, the PLRD performs a linear transformation (limited by 0 and Vk)
  • 188 on the original reference voltages, and therefore, provides the transmittance scaling policy a mechanism for adjusting the TFT-LCD transmittance function as shown in Figure 10-11a. The luminance function is shown in Figure 10-11b. 1 1 b b * t(x) = L(x) 0 gl gu 1 0 gl gu 1 x x (a) (b) Figure 10-11: (a) The LCD transmittance function (b) and the luminance function when using a programmable LCD reference driver. The similar concept of PLRD has been implemented in TFT-LCD controllers such as [12] to control contrast or gamma-correction. The PLRD represents a class of linear transformations on the backlight-scaled image. It covers both brightness scaling (adjusting gu and gl simultaneously) and contrast scaling (adjusting gu-gl). On the other hand, non-linear transformations are not desired in transmittance scaling because they cannot preserve the uniformity of contrast. 10.3.3 Contrast Fidelity The term contrast describes the concept of the differences between the dark and bright pixels. Brightness and contrast are the two most important properties of any image. In the Human Visual System [5][14], which models the perception of human vision as a three-stage processing, the brightness and contrast are perceived in the first two stages. Virtually every single display permits the users to adjust the brightness and contrast settings. For transmissive LCD monitors, the brightness control changes the backlight illumination and the contrast control changes the LCD transmittance function.
  • 189(a) Original image (b) Dim backlight to (c) Backlight (d) Transmittance 50% without luminance dimming scaling compensation Figure 10-12: Luminance functions and visual effects of adjusting brightness (b), contrast (c), and both (d) when the backlight is dimmed to 50%. Figure 10-12 shows how the brightness and contrast control change the luminance function and their visual effects when the maximum brightness is limited to 50%. Figure 10-12a depicts the original image of the USC girl. In Figure 10-12b, when the backlight is reduced to 50%, the image contrast is noticeably reduced. If we compensate for the contrast loss as shown in Figure 10-12c, then the darker (<50%) pixels will preserve their original brightness while the brighter (>50%) pixels will overshoot completely (there will be no contrast present among these pixels.) Figure 10-12d shows how the transmittance scaling generates a better image by balancing the contrast loss and number of overshot pixels. The luminance functions in Figure 10- 12b and Figure 10-12d represent the following class of linear transformations that can be implemented by the PLRD as expressed in (9): 0, 0≤ x ≤ gl −d gl = b ⋅ t ( x ) = cx + d , gl ≤ x ≤ gu , where c b−d . (10) b, gu ≤ x ≤1 gu = c Here (gl,0) and (gu,b) are the points where y=cx+d intersects y=0 and y=b, respectively. The luminance function consists of three regions: the
  • 190 undershot region [0,gl], the linear region [gl,gu], and the overshot region [gu,1]. In other words, the gl and gu are the darkest and the brightest pixel values that can be displayed without contrast distortion (overshooting or undershooting). Notice that the slope of the linear region is very close to that of the original luminance function, which is unity. The image has very few pixels in the undershot and overshot regions. Its histogram is shown in Figure 10-13a. The kernel of transmittance scaling is to find the dissimilarity between the original and backlight-scaled images, which can be solely determined by examining the luminance function b.t(x). We define the contrast fidelity function as the derivative of b.t(x): 0, 0≤ x < gl fc ( x ) = c, gl ≤ x ≤ gu, 0≤ c ≤1 (11) 0, gu < x ≤1 . c is limited between 0 and 1. If c>1, the contrast increases and deviates from that of the original image while the dynamic range [gl,gu] shrinks. The overall contrast fidelity decreases from this point, so we do not include c>1 in our solution space. The contrast fidelity is defined without quantifying contrast itself, which has no universal definition [15] and cannot help solve the optimal transmittance scaling policy problem. However, the definition of contrast fidelity does convey the concept of the classical definitions of contrast such as Webers or Michelsons that express contrast as the ratio of the luminance difference to the maximum luminance [5][14][15]. If the normalized image histogram providing the probability distribution of the occurrence of pixel value x in the image is given as: p(x)∈[0,1], x=0..255, (12) Then the global contrast fidelity of the backlight-scaled image will be defined as: gu FC = f c ( x ) ⋅ p( x ). (13) gl Fc is a function of p, gl and gu. Finding the optimal solution that minimizes Fc is called the optimal transmittance scaling policy problem.
  • 191More precisely, given the image, we would like to find the optimal backlightfactor and the PLRD transformation function, which maximize the globalcontrast fidelity. The variables of optimization are b, gu, and gl. The global contrast fidelity captures the brightness distortion due tobacklight luminance dimming, also. When the backlight is dimmed, thedynamic range [gl,gu] is shrunk accordingly, so that more pixels havecontrast fidelity of zero. Figure 10-13: (a) Histogram of the example image, (b) Optimal values of gl (leftcurve) and gl+dr (right curve) as functions of dynamic range dr, (c) Global contrast fidelity Fc as a function of dynamic range dr for b=1 (upper curve ) and b=0.5 (lower curve), (d) Optimal solutions < Fc,Pbacklight>, (e) After transmittance scaling with 10% contrast distortion (f) After backlight luminance dimming.10.3.4 Transmittance Scaling To simplify the optimal transmittance scaling policy problem, our
  • 192 approach is first to find the optimal linear transmittance function for each given backlight factor. This problem is called the contrast fidelity optimization problem. In this simplified version, given the image and backlight factor, we would like to find the optimal PLRD transformation function that maximizes the global contrast fidelity. The variables of optimization are gu and gl. Next, we sweep the backlight factor domain to find the globally optimal solutions. Our goal is to find the optimal gl and gu that maximize the overall contrast fidelity Fc. After that, the optimal coefficients c and d can be calculated from (10). The optimal transmittance function t(x) that should be applied to the LCD can then be determined as: 0, 0≤ x < gl cx + d t( x) = , gl ≤ x ≤ gu , (14) b 1, gu < x ≤1 Clearly the backlight should be concurrently dimmed to b. The optimal solution to the contrast fidelity optimization problem for an arbitrary histogram can be found by the following procedures. Let dr=gu-gl denote the required dynamic range [gl,gu] and the backlight factor b denote the available dynamic range [0,b]. For each dr, we can find the required gl + dr dynamic range [gl,gl+dr] that maximizes p( x ) . The optimal gl is found gl by scanning gl=0*256/k, 1*256/k,…(k-1)*256/k, where k represents the resolution of the PLRD in (9). Based on the histogram shown in Figure 10- 13a, Figure 10-13b shows the optimal gl and gl+dr in the x axis as functions of dr in the y axis. The left and right curves are the optimal gl and gl+dr, respectively for different dr values. This means that when the backlight is dimmed to dr, by using the available dynamic range [0,dr] from the transmissive LCD to display the required dynamic range [gl,gl+dr] by the image, we are able to generate a backlight-scaled image that minimizes the number of undershot or overshot pixels. Now consider the contrast fidelity c in (14). If the available dynamic range is larger or equal to the required dynamic range (dr b), the optimal contrast fidelity c=1 can be obtained with d 0 and the overall contrast fidelity Fc is gu simply p( x ) . Otherwise, if dr>b, the highest possible contrast fidelity is gl c=b/dr with t=1 and d=0. Thus, Fc becomes:
  • 193 b gl + dr p ( x ). (15) dr gl Figure 10-13c shows Fc as a function of dr for b=1 (upper) and b=0.5(lower). The Fc increases as dr increases from dr=0 to dr=0.5. For the b=1curve, the example image needs no more than 70% of available dynamicrange to represent the whole histogram with the best contrast fidelity c=1.For the b=0.5 curve, the Fc decreases from dr=0.5 to dr=1 because in (15) gl + drthe p( x ) increases slower than dr. The optimal Fc happens at dr=0.5 and glthe contrast fidelity c=1 in the region [gl,gl+dr]. Notice that c=1 is notalways the optimal solution when dr>b. If the distribution in the histogram isnot normal (e.g. has two peaks) the optimal dr can be greater than b, such gl + drthat p ( x ) can be increased. For each backlight factor b, the complexity of glfinding the optimal Fc, gl and gu is O(k2) with a small k (<12). Given the solution to the contrast fidelity optimization problem for anybacklight factor b, the optimal transmittance scaling policy problem can besolved by sweeping the backlight factor range between bmin and bmax, wherebmin and bmax are user-specified minimum and maximum backlight factors,respectively. All of the optimal solutions are recorded along with their powerconsumptions. The inferior solutions, i.e., those with higher or equal powerconsumptions but lower fidelity, are discarded. The remaining solutions arestored for the transmittance scaling policy to select the most suitable solutionaccording to the user preferences. Figure 10-13d shows the optimal solutionsfor b=0.8, 0.7,…0.2 from top-right to bottom-left. The x and y coordinates ofeach solution indicate the global contrast fidelity and backlight power,respectively. The two inferior solutions for b=1.0 and 0.9 are discardedbecause they have the same fidelity, Fc=1, as that of b=0.8. Results showthat more than 50% power savings can be achieved by the transmittancescaling policy while maintaining almost 100% of contrast fidelity at abacklight factor of 70%. The visual effect is shown in Figure 10-13e, incomparison with Figure 10-13f generated from the brightness-invariantpolicy from (8). A pseudo-code transmittance scaling procedure is provided below.
  • 194 transmittance_scaling(p[0..255],k) { cdf[0]=p[0]; for (i=0; i<256; i++) cdf[i]+=p[i]; for (b=bmin; b<=bmax; b+=(1/k)) { Pb=Pbacklight(b); for (dr=1; dr<=255; dr+=(256/k)) { Rmax=-1; for (g=0; g<=255-dr; g+=(256/k)) { R=cdf[g+dr]-cdf[g]; if (R>Rmax) { gl=g; Rmax=R; } } } if (b>=dr) Fc=R; else Fc=(b/dr)*R; gu=gl+dr; Sol = <Fc,Pb,b,gl,gu>; Search solution database for <Fc,*,*,*> and <*,Pb,*,*>; if (Sol is not inferior) Insert Sol into solution database; } } We use a set of benchmark images from the USC SIPI Image Database (USID) [16]. The USID is considered the de facto benchmark suite in the signal and image processing research field [5]. The results reported here are from 8 color images from volume 3 of USID. All of these images are 256 by 256 pixel images. The color depth is 24 bits, i.e., 8 bits per color-channel in the range of 0-255. Tables 10-1 and 10-2 show the results of the optimal transmittance scaling policies for the benchmarks. We use 90% as the global contrast fidelity threshold to find the minimum backlight factor and the optimal transmittance transformation. The results show an average of 3.7X savings within 10% of contrast distortion.
  • 195Table 10-1. Optimal transmittance scaling solutions to the USID benchmark images. Image Backlight Contrast Brightness Overall CCFL factor fidelity shift fidelity Power # b c d Fc (mW) 4.1.01 0.51 1 0.00 0.91 803.84 4.1.02 0.38 1 0.00 0.91 549.99 4.1.03 0.65 1 0.00 0.91 1077.21 4.1.04 0.75 1 0.00 0.91 1272.47 4.1.05 0.75 1 0.01 0.91 1272.47 4.1.06 0.84 1 0.04 0.90 1448.21 4.1.07 0.71 1 0.06 0.90 1194.36 4.1.08 0.72 1 0.04 0.92 1213.89Table 10-2. Original images (upper images) vs. transmittance-scaled images (lower images).
  • 196 10.4 SUMMARY This chapter presented a technique for reducing the backlight illumination while compensating for the luminance loss by adjusting the transmittance function of the TFT-LCD panel. First, background information about CCFL and TFT-LCD monitors was presented. Next, a contrast distortion metric to quantify the image quality loss after dimming backlight was described. Finally, the transmittance scaling problem was precisely formulated and solved. Experimental results showed that an average of 3.7X power saving can be achieved with a mere 10% contrast distortion. The proposed transmittance scaling technique was applied to still images only. However, the basic technique can be extended to video streams. For video, the decision about the backlight scaling factor is made for each frame one at a time. Consequently, the backlight factor may change significantly across consecutive frames. The enormous change in the backlight factor may introduce inter-frame brightness distortion to the observer. Therefore, when the transmittance scaling technique is applied to video applications such as an MPEG2 decoder, the inter-frame change in the backlight dimming factor should be controlled carefully such that the change is hardly noticeable to human eyes. References [1] T. Simunic et al., “Event-driven power management,” IEEE Tran. Computer-Aided Design of Integrated Circuits and Systems, vol. 20, pp. 840-857, July 2001. [2] I. Choi, H. Shim, and N. Chang, “Low-power color TFT LCD display for hand-held embedded systems,” Proc. of Symp. on Low Power Electronics and Design, Aug. 2002, pp. 112-117. [3] ANSI/IES. 1986. Nomenclature and Definitions for Illuminating Engineering, ANSI/IES RP-16-1986. New York, NY: Illuminating Engineering Society of North America. [4] Radiosity: A Programmer’s Perspective by Ian Ashdown, © October 2002 by Heart Consultants Limited. (Originally published by John Wiley & Sons in 1994.) [5] W. K. Pratt, Digital Image Processing, Wiley Interscience, 1991. [6] Maxim, MAX1610 Digitally Controlled CCFL Backlight Power Supply. [7] J. Williams, “A fourth generation of LCD backlight technology,” Linear Technology Application Note 65, Nov. 1995. [8] LG Philips, LP064V1 Liquid Crystal Display. [9] Stanley Electric Co., Ltd., [CFL] cold cathode fluorescent lamps, 2003. [10] Minolta, Minolta Precision Luminance Meter LS-100.
  • 197[11] Analog Devices, AD8511 11-Channel, Muxed Input LCD Reference Drivers.[12] Hitachi, HD66753 168x132-dot Graphics LCD Controller/Driver with Bit-operation Functions, 2003.[13] H. Aoki, “Dynamic characterization of a-Si TFT-LCD pixels,” HP Labs 1996 Technical Reports (HPL-96-19), February 21, 1996.[14] S. Daly, “The visible differences predictor: an algorithm for the assessment of image fidelity,” Digital Images and Human Vision, pp. 179-206, Cambridge: MIT Press, 1993.[15] E. Peli, “Contrast in complex images,” J. Opt. Soc. Amer. A, vol. 10, no. 10, pp. 2032- 2040, Oct. 1990.[16] A. G. Weber, “The USC-SIPI image database version 5,” USC-SIPI Report #315, Oct. 1997. Also http://sipi.usc.edu/services/database/Database.html.[17] Toshihisa Tsukada, TFT/LCD, Liquid-Crystal Displays Addressed by Tin-Film Transistors, Amsterdam: Gordon and Breach Publishers, 1996.[18] W. C. OMara, Liquid crystal flat panel displays: manufacturing science & technology, New York: Van Nostrand Reinhold, 1993.
  • 198Chapter 11POWER-AWARE NETWORK SWAPPING FORWIRELESS PALMTOP PCSAndrea Acquaviva, Emanuele Lattanzi and Alessandro BoglioloUniversit` di Urbino aAbstract Virtual memory is considered to be an unlimited resource in desktop or notebook computers with high storage memory capabilities. However, in wireless mobile devices like palmtops and personal digital assistants (PDA), storage memory is limited or absent due to weight, size and power constraints. As a consequence, swapping over remote memory devices can be considered as a viable alternative. Nevertheless, power hungry wireless network interface cards (WNIC) may limit the battery lifetime and application performance if not efficiently exploited. In this chapter we explore performance and energy of network swapping in compar- ison with swapping on local microdrives and flash memories. Our study points out that remote swapping over power-manageable WNICs can be more efficient than local swapping and that both energy and performance can be optimized through power-aware reshaping of data requests. Experimental results show that application-level prefetching can be applied to save up to 60% of swapping energy while also improving performance.Keywords: Memory management, power management, remote memory swapping.11.1 INTRODUCTION Mass storage devices provide to desktop and laptop computers the supportto implement a virtual memory that can be viewed as an unlimited resource tobe used to extend the main memory whenever needed. However, in wirelessmobile devices like palmtops and personal digital assistants (PDAs), storagememory is limited or absent due to weight, size and power constraints, thuslimiting the application of virtual memory. On the other hand, if a wirelessnetwork interface card (WNIC) is available, unlimited swapping space couldbe found on remote devices made available by a server and managed by theoperating system as either network file systems (NFS) or network block devices
  • 199(NBD). However, swapping over a power hungry WNIC may limit the batterylifetime and application performance if not efficiently exploited. In this chapter we report the results of extensive experiments conducted toevaluate and optimize the performance and power efficiency of different localand remote swap devices for wireless PDAs (namely, a compact flash (CF), amicro drive (HD) and two different WNICs). The contribution of the chapter isthree-fold. First we characterize all swap devices in terms of time and energyinherently required to swap a single page. Second, we test the effectivenessof the dynamic power management (DPM) support made available by eachdevice. Third, we show that dummy data accesses can be preemptively insertedin the source code to reshape page requests in order to significantly improvethe effectiveness of DPM. Experimental results show that WNICs are less efficient than local devicesboth in terms of energy and time per page. However, the DPM support pro-vided by WNICs is much more efficient than that of local micro drives, makingnetwork swapping less expensive than local swapping for real-world applica-tions with non-uniform page requests. Finally, we show that application-levelreshaping of page requests can be used in conjunction with DPM to save up to60% of energy while improving performance. The rest of this chapter is organized as follows. In Section 11.2 we providesome background on remote storage space. In Section 11.3 we discuss the keyfeatures of local and remote storage devices that can be used for swapping, andwe briefly outline the software support for remote swapping provided by Linux.In Section 11.4 we describe the experimental setup used for our experiments. InSection 11.5 we outline the benchmarks used to characterize each swap devicein terms of power and performance and we present characterization results. InSection 11.6 we discuss the support for dynamic power management providedby each swap device and we propose an application-level prefetching strategyfor energy-aware swapping. In Section 11.7 we report and discuss the resultsof extensive experiments conducted on a simple case study. In Section 11.8 wedraw conclusions.11.2 REMOTE STORAGE SPACE The concept of remote storage has been exploited by deeply networked sys-tems for mainly three reasons: to provide extra storage space, to enable filesharing and to enhance swapping capabilities. First, remote memories or magnetic disks are used to store application anddata by systems with limited or absent local mass storage space. Disklessworkstations and mobile terminals are both computer systems characterized bylimited or absent disk capacity.
  • 200 Even if the memory is not a constraint, remote storage space is used as repos-itory of data shared among different users working on different machines, asin the case of file servers. Access to remote data can be controlled by networkfile systems such as NFS. However, mobile networks require suitable proto-cols to handle disconnected and weakly connected operations. To this purpose,dedicated file systems and file hoarding methods have been designed [13, 7,8]. File hoarding is the technique of preparing disconnections by caching crit-ical data. Differently from traditional caching, the cost of a miss (or failure)can be catastrophic if it occurs when the system is disconnected from the net-work. To identify critical data, LRU policies augmented with user-specifiedhoard-priority have been proposed as part of the CODA file system [13]. Here,priorities are used to offset the LRU age of an object. In addition, the user isgiven the possibility to interactively control the hoarding strategy (the so calledtranslucent caching concept). Automated hoarding methods have been alsorecently presented [8, 7]. Remote memories are also commonly used as swap areas to temporarilypark run-time data and application code when the total amount of availablesystem memory is not enough to contain user processes. In computer clustersremote swap areas are designed to replace local swap partitions for performancereasons. In fact, high speed links may provide faster access than local magneticdisks especially under certain workload conditions, due to the high rotationallatency [12, 6]. While for remote file systems the main issues is reliability, forremote swapping the performance of data transfer is the key point. For thisreason, simpler and more efficient supports have been proposed [5]. Remote swap areas can be also exploited by mobile devices, where localstorage space is limited and expensive [3]. However, network swapping inmobile devices does not come for free since they are much more bandwidth andenergy constrained than desktop PCs and workstations. Remote swapping for handheld computing systems is a recent research topicthat has not been extensively studied so far. The problem of energy consumptionof network swapping in mobile devices has been faced by Hom et al. [9]. Theyproposed a compilation framework aimed at reducing the energy by switchingthe communication device on and off by means of specific instructions insertedat compile time based on a partial knowledge of the memory footprint of theapplication.11.3 SWAP DEVICES We refer to the page-based swapping support provided by the Linux OS.Linux performs a page swap in two situations: i) when a kernel daemon, ac-tivated once per second, finds that the number of free pages has fallen belowa given threshold; ii) when a memory request cannot be satisfied. The page
  • 201to be swapped-out is selected in a global way, independently from the pro-cess that made the request. The page replacement algorithm is based on anapproximation of least recently used (LRU) policy [4]. Modern operating systems equipping palmtops and PDAs make possible todefine heterogeneous support for swapping. Swapping can be performed bothlocally to the PDA and remotely, by exploiting server storage capabilities andnetwork connections. More than one swap units can be enabled at the sametime, with assigned priority. The unit with the highest priority is selected bydefault until it becomes insufficient.11.3.1 Local Devices On-board non-volatile memory is usually available in palmtop PCs to storethe bootloader and the filesystem. Magnetic disks can be added to extend filestorage capabilities. Swap can be made locally in palmtops as in desktop PCs.A dedicated partition can be defined in hard drives or flash memories, wherethe filesystem resides. Alternatively, some OS’s allow the user to define a swapfile that does not need a dedicated partition. Either way, the swap area comesat the price of decreasing the space available for actual storage purposes.Compact Flash Palmtop PCs are equipped with on-board flash memories, but additionalmemory chips can be installed as an expansion if an external slot is present.Memory Technology Device (MTD) drivers allow to define swap partitions orswap files on flash memories. However, being read-most devices, flash mem-ories are not the ideal support for swapping. Nevertheless, we evaluate theirswapping performance since they are always present in palmtop PCs, beingsometimes the only alternative to network swapping.Hard Disk Today’s technology made available hand-sized magnetic disks (called minior micro drives) suitable to be installed in palmtop computers. Currently theyprovide a storage capability up to 5GBytes. Like traditional hard disks (HD),micro drives provide a seek time much longer than the access time to sequentialblocks. For this reason, access to these kind of devices is usually performedin bursts whenever possible by exploiting on-board hardware buffers in orderto compensate for the initial transfer delay. The OS tries to limit the delaysby filtering disk accesses using software caches, whose size is limited by theavailable space in main memory. When a micro drive used as a swap device,this trade-off is even more critical, since increasing the memory space allocatedfor caching increases the number of swap requests.
  • 20211.3.2 Network Devices In order to provide the performance required to fully exploit the channelbandwidth, remote swap files can be mapped in the main memory of a remoteserver. This is the choice we made for our experiments.Network File System NFS (Network File System) is used in a network to enable file sharing amongdifferent machines on a local area. The communication protocol is based on aUDP stack, while data transfers between NFS server and clients are based onRemote Procedure Calls (RPCs). The idea of using NFS to support networkswapping is relatively recent [16]. To this purpose, a remote file must be con-figured as a swap area. This is made possible by modern operating systems thatallow the user to specify either a device or a file as a swap unit.Network Block Device A Network Block Device (NBD) offers to the OS and to the applicationsrunning on top of it the illusion of using a local block device, while data arenot stored locally but sent to a remote server [5]. As in case of NFS, the virtuallocal device is mapped in a remote file, but the swap unit is viewed as a device,rather than as a file. This is made possible by a kernel level driver (or module) that communicatesto a remote user-level server. The first time the network connection is set-up, aNDB user-level client negotiates with the NBD server the size and the accessgranularity of the exported file. After initialization, the user-level NBD clientdoes not take part to remaining transactions, that directly involve the kernel NBDdriver and the NBD server. No RPCs are required in this case, thus reducingthe software overhead. Latest releases of NBD driver use an user level networkcommunication, which affects the performance of the protocol, since data mustbe copied from the kernel to the user space address, but increases flexibility.Differently from NFS, the underlying network stack is TCP instead of UDP.This increases the reliability of network transfers, at the cost of increasing theprotocol overhead.11.4 EXPERIMENTAL SETUP We performed our experiments on a HP’s IPAQ 3600 handheld PC, equippedwith a Strong-ARM-1110 processor, 32MB SDRAM and 16MB of FLASH.Our benchmarks were executed on the palmtop on top of the Linux operatingsystem, Familiar release 6.0. The WNICs used to provide network connectivitywere a LUCENT (hereafter denoted by NICLucent) and a CISCO AIRONET350(NICCisco ), while the AP connected to the remote swapping server was a
  • 203/*************** Benchmark 1 ***************/double A[ROW][COL];initialize(A,ROW,COL);t0 = time();read_by_column(A,ROW,COL);t1 = time();/*******************************************/ Figure 11.1. Pseudo-code of the benchmark used to characterize swap devices.CISCO 350 Series base station [19, 18, 17]. The remote server was installed ona Athlon 4 Mobile 1.2 GHz notebook. For local swap experiments we used a340 MB IBM Microdrive (HD) and a 64 MB Compaq-Sundisk Compact FlashMemory (CF) [20, 21]. Power consumption of both WNICs and local deviceswas measured using a Sycard Card Extender that allowed us to monitor thetime behavior of the supply current drawn by the card. The current waveformswere then digitized using a National Instruments Data Acquisition Board PCI6024E [11] connected to a PC. A Labview (version 6.1) [10] software runningon the PC was used to coordinate the acquisition and bufferize current samplesto compute power and energy consumption. The remote swap NBD server was instrumented in order to collect time-stamped traces of swapping activity during benchmarks execution.11.5 CHARACTERIZATION OF SWAPPING COSTS To characterize the inherent cost of a page swap we developed a suite ofbenchmarks accessing data structures much larger than the available main mem-ory, without performing any computation on them. This kind of benchmark issuitable to characterization swapping cost since the computation time is neg-ligible with respect of the time spent in swapping and the devices under char-acterization are always busy serving page requests. The pseudo-code of thebenchmark is shown in Figure 11.1. A large matrix is allocated and initial-ized and then read by column in order to maximize the number of page faults.Different benchmarks were generated by changing the number of columns androws of the matrix in order to change the number of page faults while keep-ing the total size of the matrix unchanged. This allowed us to cross-validateexperimental results and reduce characterization errors. A second set of benchmarks was obtained by replacingthe read by column procedure with a write by column procedure, and usedto characterize the swapping cost in case of write-back.
  • 204Characterization Results Experimental results are reported in Table 11.1 in terms of time, energy andpower required by each local and remote device to swap a page of 4096 bytes.Both read-only and write-back results are reported. In general, write-back doubles the cost (in energy and time) of a read-onlyswap, since it involves two data transfers. As expected, local devices are moreefficient than WNICs and CF has an energy-per-page more than 10 times lowerthan all other devices. It is also worth noting that, for a given WNIC, NBD provides greater perfor-mance than NFS, at the cost of a slightly higher power consumption. Since thetime reduction overcomes the additional power consumption, the energy perpage required by NBD is lower than that required by NFS.Table 11.1. Power consumption and performance of local and remote swap devices. Swap device Read-only Write-back Type Mode Time Energy Power Time Energy Power [ms] [mJ] [mW] [ms] [mJ] [mW] CF local 4.1 0.201 49 8.2 0.402 49 HD local 3.0 1.911 637 6.4 4.061 637 NICCisco NBD 7.0 5.934 848 14.0 10.319 735 NICCisco NFS 8.5 6.123 720 14.6 10.516 720 NICLucent NBD 8.0 5.626 578 15.0 8.599 573 NICLucent NFS 10.0 5.243 524 22.0 10.672 48511.6 POWER OPTIMIZATION In the previous section we have characterized swap devices in terms of timeand energy requirements per swap page. To this purpose we designed a setof benchmarks that simply accessed data structures much larger than the mainmemory without performing any computation on them. Although useful for characterization purposes, the benchmarks of Figure 11.1are unrealistic for two main reasons. First, computation time is usually non-negligible, so that page requests are spaced in time according to a distributionthat depends on the workload and on the state of the main memory. Second, thetotal size of the data structures accessed by each application usually does notexceed the size of the main memory, or otherwise the performance degradationwould not be acceptable. In most cases of practical interest, swapping is mainly needed after a contextswitch to bring in main memory data structures the first time they are used bythe active process. Moreover, in handheld devices there are often only a fewprocesses running concurrently, so that both main memory and peripherals are
  • 205Table 11.2. Power states of local and remote devices. Device State Power Timeout WU-time WU-power [mW] [ms] [ms] [mW] CF Read 107 Write 156 Wait 4.5 HD Read 946 Write 991 Wait 600 Sleep 24 2000 4500 ± 1980 1067 NIC Receive 755 (cisco) Transmit 1136 Wait 525 (PSP/PSPCAM) 113 0/850 14/14 400 Power-Off 0 any 370 451 NIC Receive 548 (lucent) Transmit 798 Wait 407 (PSP) 38 100 1 800 Power-Off 0 any 270 357mainly used by a single process at the time. In this situation, the usage pattern ofswapping devices are significantly different from those used for characterizingswapping costs because of the presence of long idle periods between pageswaps. Since swapping devices spend power while waiting for page requests, theeffective energy per page is larger than that reported in Table 11.1. On theother hand, idleness can be dynamically exploited to save power by puttingthe devices in low-power operating modes, or by turning them off. Dynamicpower management (DPM) significantly impacts the performance and energytrade-off offered by each device under bursty workloads. In this section, we first analyze the DPM supports provided by each swappingdevice, then we show how to increase their effectiveness by means of softwareoptimization techniques aimed at reshaping the distribution of page requests.11.6.1 DPM Support Evaluation The DPM support provided by each swap device is schematically representedin Table 11.2. For each device, the key features of active and inactive operatingmodes are reported. Active modes are characterized only in terms of powerconsumption, while inactive modes are also characterized in terms of timeout tobe waited before entering the inactive state, wake-up time and wake-up power.
  • 206The data reported in the Table have been obtained by analyzing the currentprofiles provided by the measurement setup described in Section 11.4. First of all we remark that the average power consumptions measured duringpage swaps (reported in Table 11.1) are not equal to the power consumptionsmeasured for the devices during read/receive or write/transmit. In fact, forinstance, a page swap across a wireless link entails the transmission of thepage request, a waiting time corresponding to the latency of the remote device,the reception of the page and, possibly, the write-back of a swapped-out page.The average swapping power comes from the weighted average of all thesecontributions. The CF has no inactive states. This is because its power consumption in waitmode is negligible, making inactive low-power states useless. On the contrary,NICs and HDs consume a large amount of power while waiting for servicerequests, so that it is worth switching them to low-power inactive states duringlong idle periods. The Sleep state of the HD has the lowest power consumption, but the highestwakeup cost in terms of power (higher than 1W) and time (in the order of severalseconds). Moreover, the wakeup time is highly unpredictable, its measuredstandard deviation being almost 2 seconds. According to the IEEE802.11b standard, WNICs provide MAC-level DPMsupport that can be enabled via software [1]. The actual implementation ofthe DPM support depends on the WNIC. The protocol policy (PSP) consists inplacing the card in a low-power state called doze mode, in which it sleeps butwakes-up periodically to keep synchronized with the network and to check theaccess point (AP) for outstanding data. A polling frame bust be transmitted bythe card for each packet to be retrieved. PSP mode provides power savings ata cost of a noticeable performance hit. To increase performance, a variation ofthis policy is implemented by CISCO cards. They automatically switch fromPSP to CAM (Constant Awake Mode) when a large amount of traffic is detected.In this case no polling frame is needed between packets since the reception andtransmission happen in active mode. Even if the power consumption in sleep state is low, it is not negligible.Moreover, the card is sensitive to broadcast traffic. A more aggressive policywould require to completely shut-off the card when no needed by any activeapplication in the system. Thus, more power can be saved, at the price of alarger wake-up delay needed by network re-association. OS-level policies canbe implemented to this purpose based on a power management infrastructurerecently developed for Linux OS [2]. This infrastructure is composed by apower manager module that handles requests from applications and keeps trackof their resource needs. On the other side, upon a request, the power managercan directly switch off a peripheral (WNIC in our case) if no other applicationsare using it. Switch off request may come from user applications through
  • 207/*************** Case study ****************/double dummy[2048][2048], C[128][128];double A[128][128], B[128][128];initialize(A,128,128);initialize(B,128,128);initialize(C,128,128);initialize(dummy,2048,2048); //swap outt0 = time();compute_product(A,B,C);t1 = time();/*******************************************/ Figure 11.2. Pseudo-code of the case study.dedicated APIs or directly by another kernel module. We exploited this featureto let the NBD driver module switch on and off the card between swappingrequests. The features of doze and power-off modes are reported for both WNICs inTable 11.2. We observe that the MAC-level DPM support of NICLucent is moreefficient that that of NICCisco , but the DPM policy is more conservative (thetimeout being 100ms).11.6.2 Re-shaping of Swap Requests The effectiveness of any DPM strategy strongly depends on workload statis-tics. Regardless of the DPM policy, the higher the burstiness of the workload,the higher the power savings. In fact, long idle periods can be effectivelyexploited to switch to the deepest inactive states, while long activity burstsamortize the cost of wake up. Although caching and buffering can be performed by the OS to perform alow-level reshaping of page requests, a typical trace of swapping traffic showssmall bursts of a few pages followed by short periods of inactivity. Increasingthe granularity of page swaps could increase the burstiness of the workload, butalso increases the risk of preemptively swapping-in unused pages. On the other hand, in many cases data pre-fetching could be deterministicallyperformed at the application-level in order to reshape swapping traffic. Thiscan be done by inserting dummy accesses to the data structures right beforethey are used. Dummy accesses generate bursts of page requests for two mainreasons: first, they are not delayed by any commutation, second, a single accessis sufficient to fetch an entire page.
  • 20811.7 CASE STUDY We use matrix multiplication as a case study to evaluate the effectiveness ofthe DPM strategies implemented by the swap devices and to demonstrate thefeasibility of application-level reshaping of swap requests. The pseudo-code of the case study is reported in Figure 11.2: it simplycomputes the product of two square matrices A and B and puts the result in a thirdmatrix C. The total size of the three matrices fits in main memory, but we use adummy matrix, exceeding the size of the physical memory, to force swappingactivity. Matrices A, B and C are first allocated and initialized, then the dummymatrix is initialized in order to swap A, B and C out from main memory. Inpractice, the initialization of the dummy matrix creates boundary conditionssimilar to those possibly caused by the execution of other applications. Thenwe monitor the execution time and the swapping energy caused by the executionof the compute product procedure. The distribution of swap requests is shown in Figure 11.3. The expecteddistribution is also plotted for comparison. The large number of pages requestedat the beginning corresponds to the upload of the entire matrix B. In fact, the firstcolumn of B has to be read in order to compute the first entry of C. Since matricesare stored in memory by rows, reading the first column entails swapping theentire matrix. Subsequent page requests are spaced in time according to thetime required to compute 512 × 128 floating point products. Comparing the actual requests with the theoretical needs, we observe thatthe OS swaps 8 pages at the time, thus increasing the opportunity for DPM.However, the total number of pages request by the OS is 104, while the threematrices fit into 96 pages. To reshape swap requests we inserted dummy accesses to the three matricesbetween the computation of initial time t0 and the computation of the matrix pro-duct. Dummy accesses were performed by a routing, called access one per page,that reads one matrix entry every 512 (i.e., one entry per page).Case Study Results Experimental results obtained by executing the case study with and withouttraffic reshaping are reported in Table 11.3. Based on the results reported inTable 11.1 we decided to use NBD for remote swapping. In Figure 11.5 wedetailed the comparison among different devices. Each device was tested with and without DPM. The two WNICs were testedwith both MAC-level DPM (doze) and OS-level DPM (power-off). The DPMmode and the corresponding timeouts are reported in the first column. Theperformance of the CF and the CPU time obtained by running the applicationwith data available in main memory are also reported in the first two rows forreference. The DPM of the HD was enabled by default, so that data reported
  • 209 105 100 95 90 85 80 75 Number of pages 70 65 Theoretical needs 60 Page swapped 55 50 45 40 35 30 25 20 15 10 5 0 0 5 10 15 20 25 30 Execution time [s] Figure 11.3. Distribution of page requests.on row HD are computed from previous characterization. All other data wereobtained from real measurements, by repeating each experiment 4 times. Interestingly, even without DPM, the HD consumes more energy than WNICs.This is because of its higher power consumption when idle. When DPM is en-abled, WNICs become much more convenient than HD. In particular, the DPMof the HD is counterproductive both in terms of time and energy under this traf-fic conditions because of the large wakeup cost. On the contrary, MAC-levelDPM of NICCisco and NICLucent saves respectively more than 50% and morethan 80% of the swapping energy. If the power-off state is exploited, powersavings become of 85% and 94%, respectively, with negligible performanceloss. When DPM policies are enabled, traffic reshaping provide further advantagesboth in terms of energy and execution time. For the HD, traffic reshaping makesDPM effective to save more than 60% of energy consumption. For WNICs,traffic reshaping provides additional energy savings while further reducing theperformance loss. The ratios between results obtained with and without trafficreshaping are reported in the table. In particular, when the power-off state isexploited, traffic reshaping leads to additional energy savings around 60%. The overall effect of DPM and traffic reshaping makes network swappingmuch more energy efficient than local swapping on a HD (the overall energybeing almost 10 times lower) with a performance penalty of about 5%. In Figure 11.4 we reported current profiles obtained by running the case studybenchmark for the Compact Flash, Cisco card without power management andmicrodrive. The Figure evidences the strong variance of microdrive’s powerprofile, due to the variability of the head position and speed at the shutdown
  • 210Table 11.3. Execution time and swapping energy required to run the case study of Figure 11.2with and without traffic reshaping. Device Exec. time [s] Original Reshaped Ratio Avg Std Avg Std RAM 25 0 25.25 0.5 1.01 CF 25.5 0.57 25.75 0.5 1.01 HD 25.31 - 25.81 - 1.01 (PM ON) 37.75 5.91 27.75 0.96 0.73 NICCisco 26 - 26.33 0.58 1.01 (PSPCAM) 30.67 2.16 27 1 0.88 (PSP) 43.33 0.58 45 - 1.04 (OFF) 28.75 0.5 26.0 0.82 0.93 NICLucent 30.25 0.5 28.5 1.0 0.94 (PSP) 30.0 0 27.75 0.5 0.92 (OFF) 30.0 0 28.25 0.5 0.94 Energy [mJ] Original Reshaped Ratio Avg Std Avg Std RAM - - - - - CF 0.14 0.003 0.16 0.02 1.12 HD 15.20 - 15.50 - 1.01 (PM ON) 19.43 5.31 6.21 0.85 0.32 NICCISCO 16.51 0.01 16.73 0.38 1.01 (PSPCAM) 10.59 0.23 6.64 0.61 0.63 (PSP) 8.05 0.23 8.83 0.45 1.09 (OFF) 2.47 0.09 0.89 0.051 0.36 NICLU CENT 13.60 0.56 12.70 0.51 0.93 (PSP) 2.54 0.096 2.19 0.08 0.86 (OFF) 1.76 0.08 0.72 0.045 0.41instants. In fact, the two bottom traces in Figure are obtained running thesame benchmark. As a reference, we marked swapping activity intervals withuppercase latin letters.11.8 CONCLUSION In conclusions, our experiments demonstrate the feasibility and the energyefficiency of network swapping from wireless palmtop PCs. The effectivenessof the DPM support provided by WNICs makes them more efficient than localHDs and open the field to optimization strategies (like swap reshaping) thatmay further improve energy efficiency and performance.
  • 211 20 Compact Flash 15 A B C D E F G 10 5 0 A B C D E F G Cisco WNIC 175 150 Current [mA] 125 100 A BC DE FG IBM uHD 200 100 0 A BC D E F G IBM uHD 200 100 0 0 10 20 30 40 Time [s]Figure 11.4. Power profiles of the different swap devices during the execution of the case studyof Figure 11.2. Figure 11.5. Comparison of energy consumptions reported in Table 11.3References[1] LAN/MAN Standards Committee of the IEEE Computer Society. Part 11: Wireless LAN MAC and PHY Specifications: Higher-Speed Physical Layer Extension in the 2.4 GHz Band, IEEE, 1999.[2] A. Acquaviva, T. Simunic, V. Deolalikar, S. Roy, "Remote Power Control of Wireless Network Interfaces," Proceedings of PATMOS, Turin, Italy, Sept. 2003.
  • 212[3] I. Bokun, K. Zielinski, "Active Badges–The Next Generation," http://www.linuxjournal.com/article.php?sid=3047, Oct. 1998. ´[4] D. Bovet, M. Cesati, "Understanding the Linux Kernel," OReally & Asso- ciates, Sebastopol, CA, Jan. 2001.[5] P. T. Breuer, A. Marin Lopez, A. Garcia Ares, "The Network Block Device," Linux Journal, Issue 73, May 2000.[6] M. D. Flouris, E. P. Markatos, "The Network RamDisk: Using Remote Memory on Heterogeneous NOWs," Cluster Computing, pp. 281-293, 1999, Baltzer Science Publishers.[7] G. H. Kuenning, G. J. Popek, “Automated Hoarding for Mobile Comput- ing,” Proc. of Symposium on Operating System Principles, pp. 264–275, Oct. 1997.[8] G. Kuenning, W. Ma, P. Reiher, G. J. Popek, “Symplifying Automated Hoarding Methods,” Proc. of MSWiM, pp. 15–21, Sept. 2002.[9] J. Hom, U. Kremer, "Energy Management of Virtual Memory on Diskless Devices," Proceedings of COLP, Barcelona, Spain, Sept. 2001.[10] National Instruments, "Labview User Manual," http://www. ni.com/pdf/manuals/320999d.pdf[11] National Instruments, "NI 6023E/6024E/6025E Family Specifications," http://www.ni.com/pdf/manuals/370719b.pdf[12] T. Newhall, S. Finney, K. Ganchev, M. Spiegel, "Nswap: A Network Swap- ping Module for Linux Clusters," Proceedings of Euro-Par, Klagenfurt, Austria, August 2003.[13] M. Satyanarayanan, "The Evolution of Coda," ACM TOCS, Vol. 20, Is- sue 2, Pages: 85–124, May 2002.[14] A. Silbershatz, P. Galvin, G. Gagne, "Operating System Concepts, 6th Edition," Addison-Wesley, 2002.[15] Sycard Technology, "PCCextend 140 CardBus Extender User’s Manual," http://www.sycard.com/docs/cextman.pdf (1996)[16] "Swapping via NFS for Linux," http://www.nfs-swap.dot-heine.de[17] Cisco System, Cisco Aironet 350 Series Access Points, http://www.cisco.com/univercd/cc/td/doc/product/wireless/ airo 350/accsspts/index.htm, 2003.[18] Cisco System, Cisco Aironet 350 Series Wireless LAN Adapters, http://www.cisco.com/univercd/cc/td/doc/product/wireless/ airo 350/350cards/index.htm, 2003.[19] Agere, 802.11 Wireless Chip Set White Paper, http://www.agere. com/client/docs/multimode white paper.pdf, 2003.
  • 213[20] IBM, 340MB Microdrive Hard Drive, http://www.storage. ibm.com/hddredirect.html?/micro/index.html, 2003.[21] Compaq, compact flash cards, http://www.hp.com/products1/ storage/products/storagemedia/flash cards/index.html, 2003.
  • 214Chapter 12ENERGY-EFFICIENT NETWORK-ON-CHIP DESIGNDavide Bertozzi1, Luca Benini1 , Giovanni De Micheli21 University of Bologna;2 Stanford UniversityAbstract Performance and power consumption of multi-processor Systems-on-Chip (SoCs) are increasingly determined by the scalability properties of the on-chip communication architecture. Networks-on-Chip (NoCs) are a promising solu- tion for efficient interconnection of SoC components. This chapter focuses on low power NoC design techniques, analyzing the related issues at different lay- ers of abstraction and providing examples taken from the most advanced NoC implementations presented in the open literature. Particular emphasis is given to application-specific NoC architectures, in that they represent the most promising scenario for minimization of communication-energy in multi-processor SoCs.Keywords: Network-on-Chip, Low Power, Micro-network Stack, Application-Specific12.1 INTRODUCTION The most critical factor in Systems-on-Chip (SoCs) integration will be relatedto the communication scheme among components. The challenges for on-chipinterconnect stem from the physical properties of the interconnection wires.Global wires will carry signals whose propagation delay will exceed the clockperiod. Thus signals on global wires will be pipelined. At the same time, theswitched capacitance on global wires will constitute a significant fraction ofthe dynamic power dissipation. Moreover, estimating delays accurately willbecome increasingly harder, as wire geometries may be determined late inthe design flow. Hence, the need for latency insensitive design is critical. Themost likely synchronization paradigm for future chips is globally-asynchronouslocally-synchronous (GALS), with many different clocks. SoC design will be guided by the principle of consuming the least possiblepower. This requirement matches the need of using SoCs in portable battery-powered electronic devices and of curtailing thermal dissipation which canmake chip operation infeasible or impractical. Whereas computation and stor-
  • 215age energy greatly benefits from device scaling (smaller gates, smaller memorycells), the energy for global communication does not scale down. On the con-trary, projections based on current delay optimization techniques for globalwires [20] show that global on-chip communication will require increasinglyhigher energy consumption. Hence, communication-energy minimization willbe a growing concern in future technologies [40]. Energy considerations will impose small logic swings and power supplies,most likely below 1 Volt. Electrical noise due to cross-talk, electro-magneticinterference (EMI) and radiation-induced charge injection (soft errors) will belikely to produce data upsets. Thus, the mere transmission of digital values onwires will be inherently unreliable. To cope with these problems, network design technology can be used to ana-lyze and design SoCs modeled as micro-networks of components (or Networks-on-Chip, NoCs). The SoC interconnect design analysis and synthesis is basedupon the micro-network stack paradigm, which is an adaptation of the pro-tocol stack [25] (Figure 12.1) used in networking. This abstraction is usefulfor layering micro-network protocols and separating design issues belonging todifferent domains. SoCs differ from wide-area networks because of local proximity and be-cause they exhibit much less non-determinism. In particular, micro-networkshave a few distinctive characteristics, namely, energy constraints, design-timespecialization and low communication latency. This chapter focuses on low power NoC design techniques, and analyzesspecific design issues related to the different layers of abstraction outlined inthe micro-network stack in a bottom-up way. The objective is to describe,for each layer, how the system interconnect is progressively abstracted andwhat the most relevant micro-network design issues are in order to come upwith an energy-efficient NoC architecture. Particular emphasis is given to cus-tomized, domain-specific NoCs, which represent the most promising scenariofor communication-energy minimization in the context of NoC-based multi-processor SoCs (MPSoCs). In most cases, specific solutions proposed in theliterature are outlined, even though it should be clear that many design issuesare open and significant progress in this area is expected in the near future.12.2 PHYSICAL LAYER Global wires are the physical implementation of the communication chan-nels. Traditional rail-to-rail voltage signaling with capacitive termination, asused today for on-chip communication, is definitely not well-suited for high-speed, low-energy communication on future global interconnect [16]. Reducedswing, current-mode transmission, as used in some processor-memory systems,
  • 216 Figure 12.1. Micro-network stackcan significantly reduce communication power dissipation while preservingspeed of data communication [29]. In the case of a simple CMOS driver, low-swing signaling is achieved bylowering the driver’s supply voltage Vdd . This implies a quadratic dynamic 2power reduction (because Pdyn = KVdd ). Unfortunately, swing reductionat the transmitter complicates the receiver’s design. Increased sensitivity andnoise immunity are required to guarantee reliable data reception. Differentialreceivers have superior sensitivity and robustness, but they require doubling thebus width. To reduce the overhead, pseudo-differential schemes have been pro-posed, where a reference signal is shared among several bus lines and receivers,and incoming data is compared against the reference in each receiver. Pseudo-differential signaling reduces the number of signal transitions, but also noisemargins with respect to fully differential signaling. Thus, reduced switchingactivity is counterbalanced by higher swings and determining the minimum-energy solution requires careful circuit-level analysis. Dynamic voltage scaling has been recently applied to busses [23, 26]. In[26] the voltage swing on communication busses is reduced, even though signalintegrity is partially compromised. Encoding techniques can be used to detectcorrupted data which is then retransmitted. The retransmission rate is an inputto a closed-loop DVS control scheme, which sets the voltage swing at a trade-offpoint between energy saving and latency penalty (due to data retransmission). The On-Chip Network (OCN) for low power heterogeneous SoC platformsillustrated in [9] employs some advanced techniques for low-power physical in-terconnect design. OCN consists of global links connecting clusters of tightly-connected IPs which are several millimeters long. By using overdrivers, clockedsense amplifiers and twisted differential signaling, packets are transmitted reli-ably with less than 600 mV swing. The size of a transceiver and the overdrivevoltage are chosen to obtain a 200 mV separation at the receiver end. A 5 mm
  • 217global link of 1.6um wire-pitch can carry a packet at 1.6GHz with 320ps wire-delay and consumes 0.35pJ/bit. On the contrary, a full-swing link consumes upto 3x more power and additional area of repeaters. An on-chip serialization technique [10] is also used in OCN, thus significantlyreducing area. However, the number of signal transitions on a link is increasedsince the temporal locality between adjacent packets is removed. An ad-hocserialized low-energy transmission coding scheme was therefore designed as anattempt to exploit temporal locality between packets. The encoder generates a’1’ only when there is difference between a current packet and a previous packetbefore it is serialized. The decoder then uses this encoded packet to reconstructthe original input, using its previously stored packet. A 13.4% power savingis obtained for a multimedia application. The power overhead associated withthe encoder/decoder is only 0.4mW. Nevertheless, as the technology trends lead us to use smaller voltage swingsand capacitances, the upset probabilities will rise. Thus the trend toward fasterand lower-power communication may decrease reliability as an unfortunateside effect. Reliability bounds as voltages scale can be derived from theoretical(entropic) considerations [19] and can be measured also by experiments on realcircuits. Finally, another key physical-layer issue is synchronization. In fact, globalsynchronization signals (i.e., clocks) are responsible for a significant fractionof the power budget in digital integrated systems. Alternative on-chip synchro-nization protocols that do not require the presence of a global clock have beenproposed in the past [30, 11] but their effectiveness has not been studied indetail from the energy viewpoint. In the OCN NoC [9], a programmable power management unit provides fourclocks with PLL; 1.6GHz for the OCN, 800 MHz for schedulers, 100MHz forprocessors and 50 MHz for peripherals. Those clock frequencies are scalableby software for power-mode control and also for optimal operation of eachapplication.12.3 SYSTEM INTERCONNECT ARCHITECTURE Designing the architecture for an on-chip interconnect requires choices at ahigher level of abstraction with respect to physical interconnect design, but alsowith a stronger impact on energy dissipation. Traditional shared buses try to overcome their energy inefficiency by meansof bus splitting [5]. The bus is split into smaller segments and proper bridgesare inserted to ensure communication between any two adjacent segments whenneeded. Thus, the load capacitance charged and discharged at each bus accessis reduced. Most commercial shared buses make use of this solution, includingAMBA bus [1] and IBM CoreConnect [33]. They split the bus based on the
  • 218characteristics of the connected masters and slaves (e.g. high performance coresversus slow peripherals). More advanced bus specifications (such as AMBAMulti-Layer [34]) allow to group IP cores into clusters, and this can be donebased on their interaction during application execution. As an example, OCN [9] exploits locality of IP cores by grouping theminto clusters, and a crossbar switch is used for intra-cluster packets, perform-ing buffer-less cut-through switching. A round-robin scheduling of the switchensures fairness and starvation-freedom to OCN. An n × n crossbar fabriccomprises n2 crosspoint junctions which contain NMOS pass-transistors. Ina conventional crossbar fabric, each input driver wastes power to charge twolong wires (horizontal and vertical) and 2n transistor-junction-capacitors. OCNemploys a crossbar partial activation technique. By splitting the crossbar fab-ric into 4 × 4 tiles, input and output wires can be divided into four. A gatedinput driver at each tile is activated only when the scheduler grants access tothe tile. The output signal does not propagate to other tiles to reduce the powerconsumption on the vertical wire. A 43% power saving is obtained in a 16 × 16crossbar switch fabric with a negligible area overhead.12.3.1 Network topology Energy considerations might affect the on-chip network topology selectionprocess, as showed by the architectural choices made in the design of recentlyproposed NoC solutions. Again, the OCN case is very instructive. In fact, a star topology guaranteeingconstant and minimum switch hop counts between every communicating IP wasadopted in an early implementation [10]. However, a 1-level flat star topologyresults in a number of capacitive global wires that may cause long latencyand large power dissipation. Therefore, the most recent solution consists of ahierarchical SoC composed of clusters of tightly, star-connected IPs. Octagon [21] on-chip communication architecture consists of 8 nodes and12 bi-directional links connected according to an octagonal topology. In thisway, communication between any pairs of nodes can be performed by at mosttwo hops. Moreover, Octagon exhibits higher aggregate throughput than ashared bus or crossbar interconnect, a simple, shortest past routing algorithmand less wiring than a crossbar interconnect. Octagon and OCN are examplesof network topologies that try to provide the highest degree of connectivitybetween network nodes while trying to minimize the number of hops, thereforetargeting high-performance and low-power NoC realizations. Power-aware topology selection is briefly discussed in [31] with respect to theSoCIN NoC architecture. A mesh topology is compared with a torus one: theformer exhibits lower costs, while the latter reduces message latency. To avoidthe long wrapping-around links, with a very high associated capacitive load,
  • 219a folded torus topology can be used [13]. Such approach reduces the wiringlengths and the power consumption while allowing to improve the operatingfrequency of network channels. A more detailed comparison between the power efficiency of a mesh and afolded torus topology is addressed in [13]. The power has been decomposedinto the power per hop (traversal of input and output controllers) and powerper wire distance travelled. The analysis shows that if wire transmission powerdominates per-hop power, the mesh is more power efficient. For the 16 tilenetwork considered in [13], the wire transmission power was estimated to besignificantly greater than per-hop power, however the power overhead of thetorus was small (less than 15%), and was counterbalanced by the benefits of itslarger effective bandwidth.12.4 DATA LINK LAYER The data-link layer abstracts the physical layer as an unreliable digital link,where the probability of bit upsets is non null (and increasing as technologyscales down). Furthermore, reliability can be traded off for energy [19]. Themain purpose of data-link protocols is to increase the reliability of the link upto a minimum required level, under the assumption that the physical layer byitself is not sufficiently reliable. An effective way to deal with errors in communication is to packetize data. Ifdata is sent on an unreliable channel in packets, error containment and recoveryis easier, because the effect of errors is contained by packet boundaries, anderror recovery can be carried out on a packet-by-packet basis. For the realization of on-chip micro-networks, several error recovery mech-anisms developed for macroscopic networks can be deployed, but their energyefficiency should be carefully assessed in this context. As a practical example,consider two alternative reliability-enhancement techniques: error-correctingcodes and error-detecting codes with retransmission. A set of experimentsinvolved applying error correcting and detecting codes to an AMBA bus andcomparing the energy consumption in four cases [12]: 1) original unencodeddata; 2) single-error correction, 3) single-error correction and double-error de-tection, 4) multiple-error detection. Hamming codes were used. Note that incase 3, a detected double error requires retransmission. In case 4, using (n, k)linear codes, 2n − 2k errors patterns of length n can be detected. In all cases,some errors may go undetected and be catastrophic. Using the property of thecodes, it is possible to map the mean time to failure (MTTF) requirement intobit upset probabilities, and thus comparing the effectiveness of the encodingscheme in a given noisy channel (characterized by the upset probability) inmeeting the MTTF target.
  • 220 4600 46 0.46 0.0046 4.6e-5 4.6e-7 4.6e-9 MTTF (days) Figure 12.2. Energy efficiency for various encoding schemes The energy efficiency of various encoding schemes varies: we summarizehere one interesting case, where three assumptions apply. First, wires arelong enough so that the corresponding energy dissipation dominates encod-ing/decoding energy. Second, voltage swing can be lowered until the MTTFtarget is met. Third, upset probabilities are computed using a white Gaussiannoise model [18]. Figure 12.2 shows the average energy per useful bit as afunction of the MTTF (which is the inverse of the residual word error proba-bility). In particular, for reliable SoCs (i.e., for MTTF = 1 year), multiple-errordetection with retransmission is shown to be more efficient than error-correctingschemes. We refer the reader to [12] for results under different assumptions. Another important aspect affecting the energy consumption is the media ac-cess control (MAC) function. Currently, centralized arbitration schemes arewidely adopted [1, 14] for the serialization of bus access requests. Unfortu-nately, central arbiters are instance-specific and therefore poorly scalable. Infact, the energy cost of communicating with the arbiter, and hardware complex-ity of the arbiter itself scale up more than linearly with the number of bus masters.The selection of a specific arbitration algorithm impacts both performance andpower consumption [32]. Alternative multiplexing approaches, such as code di-vision multiplexing, are actively investigated for on-chip communication [27].However, research in this area is just burgeoning, and significant work is neededto develop energy-aware media-access-control for future micro-networks. Arbitration mechanisms are required also in the implementation of NoCswitches to address contention resolution problems such as: prioritizing oneout of multiple input channels whose packets have to be directed to the same
  • 221output channel or multiplexing multiple virtual channels onto the same physicaloutput link.12.5 NETWORK LAYER At the network layer, packetized data transmission can be customized by thechoice of switching and routing algorithms. The former establishes the type ofconnection while the latter determines the path followed by a message throughthe network to its final destination. In circuit switching data and control are separated: control is provided to thenetwork just to set up a connection over which all subsequent data is transportedin a connection-free fashion. On the contrary, packet-switched on-chip networks naturally offer best ef-fort services, as contention takes place at the granularity of individual packets.Packet arrival cannot be predicted and contention has to be resolved dynami-cally: power-hungry data storage is required at the routers for this purpose, andthe provision of guarantees is complicated. However, a better link utilization isachieved and error control is made easier. The most promising packet switching technique for NoC application is worm-hole switching. It was originally designed for parallel computer clusters [17]because it achieves the minimal network delay and requires fewer buffers. Inwormhole switching, each packet is further segmented into flits (flow controlunit). The header flit reserves the routing channel of each switch, the body flitswill then follow the reserved channel, the tail flit will later release the channelreservation. One major advantage of wormhole switching is that it does not require thecomplete packet to be stored in the switch while waiting for the header flit tobe routed to the next stages. Wormhole switching not only reduces the store-and-forward delay at each switch, but it also requires much less buffer spaces.Because of these advantages, wormhole switching is an ideal candidate tech-nique for on-chip interconnect networks [13], although deadlock and livelockare potential problems that need to be taken care of [17, 15]. Routing algorithms can be static (packets injected into the network alreadyinclude routing information and only minimum header processing is requiredat the switches) or dynamic (routing decisions are dynamically taken at theswitches). These latter policies allow packet routes to adapt to network condi-tions, and therefore trade-off the energy savings obtained in this way with theincreased switch complexity and related energy dissipation. Next, a compari-son between energy efficiency of routing techniques is provided as an exampleof network-level design decisions for low power.Contention-Look-Ahead Routing A contention-look-ahead routing schemeis the one where the current routing decision is helped by monitoring the adjacent
  • 222switches, thus possibly avoiding or reducing blockages and contention in thecoming stages. A contention-aware routing scheme is described in [22]. The routing decisionat every node is based on the “stress values” (the traffic loads of the neighbors)that are propagated between neighboring nodes. This scheme is effective inavoiding “hot spots” in the network. The routing decision steers the packets toless congested nodes. To solve the contention problems in wormhole switching schemes, acontention-look-ahead routing algorithm can be used, which “foresees” thecontention and delays in the coming stages using a direct connection from theneighboring nodes. The major difference from [22] is that information is han-dled in flits, and thus large and/or variable size packets can be handled withlimited input buffers. Furthermore, because it avoids contention between pack-ets and requires much less buffer usage, the latter contention-look-ahead routingscheme can greatly reduce the network power consumption. At every intermediate stage, there may be many alternate routes to go tothe next stage. We call the route that always leads the packet closer to thedestination a profitable route. Conversely, a route that leads the packet awayfrom the destination is called misroute [17]. In mesh networks, profitable routesand misroutes can be distinguished by comparing the current node ID with thedestination node ID. Profitable routes will guarantee a shortest path from source to destination.Nevertheless misroutes do not necessarily need to be avoided. Occasionally,the buffer queues in all available profitable routes are full, or the queues are toolong. Thus detouring to a misroute may lead to a shorter delay time. Underthese circumstances, a misroute may be more desirable. It is interesting to compare the contention-look-ahead routing algorithm withdimension order routing – a routing scheme that always routes the packets on onedimension first, upon reaching the destination row or column, then switches tothe other dimension until reaching the destination. Dimension ordered routingis deterministic and guarantees shortest path, but it cannot avoid contention. The contention-look-ahead routing will reduce the power consumption onthe buffers because it can “foresee” the contention in the forthcoming stages andshorten the buffer queue length. On the contrary, dimension-ordered routingalways steers the packets along the shortest path, while contention-look-aheadrouting may choose the misroute when contention occurs and therefore hasa larger average hop count per packet. This translates to more power on theinterconnect. Finally, the contention-look-ahead routing switch needs more logic gatesthan dimension-ordered routing. However, simulation results show that with16 RISC processors on a 4x4 mesh interconnect, contention-look-ahead rout-
  • 223 Figure 12.3. Cache and Memory Energy Decrease as Packet Payload Size Increasesing reduces the total network power by about 15% with 16-flit buffers. Thereduction is more significant with larger buffer sizes.12.6 TRANSPORT LAYER At the transport layer, algorithms deal with the decomposition of messagesinto packets at the source and their assembly at destination. The choice ofinformation decomposition into packets or flits, as well as the choice of packetsize can heavily impact energy efficiency. Next, we will use the shared-memoryMPSoC as a case study to analyze the packet size trade-offs both qualitativelyand quantitatively. The system architecture consists of an on-chip intercon-nect which provides connectivity to nodes composed by a RISC processor, itscaches, a local memory reachable by means of a local bus and the network in-terface. The MPSoC power consumption originates from three sources: 1) thenode processor power consumption, 2) the cache and shared memory powerconsumption, and 3) the interconnect network power consumption. We willstart first from the cache and memory analysis.Cache and memory power consumption Whenever there is a cache miss,the cache block content needs to be encapsulated inside the packet payloadand sent across the network. In shared-memory MPSoC, the cache block sizecorrelates with the packet payload size. Larger packet sizes will decrease thecache miss rate, because more cache content can be updated in one memoryaccess. Consequently, both cache energy consumption and memory energyconsumption will be reduced. This relationship can be seen from Fig. 12.3.It shows the energy consumption of cache and memory under different packetsizes. The energy in the figure is normalized to the value of 256Byte, whichachieves the minimum energy consumption.
  • 224Figure 12.4. Network and Total MPSoC Energy Consumption under Different Packet PayloadSizesInterconnect network power consumption The power consumption ofpacketized dataflow on MPSoC network is determined by the following threefactors: 1) the number of packets on the network, 2) the energy consumed byeach packet on one hop, and 3) the number of hops each packet travels. Wesummarize these effects and list them below: 1 Packets with larger payload size will decrease the cache miss rate and consequently decrease the number of packets on the network. 2 Larger packet size will increase the energy consumed per packet, because there are more bits in the payload. 3 Larger packets will occupy the intermediate node switches for a longer time, and cause other packets to be re-routed to non-shortest paths. This leads to more contention that will increase the total number of hops needed for packets traveling from source to destination. Actually, increasing the cache block size will not decrease the cache missrate proportionally. Therefore, the decrease of packet count cannot compensatefor the increase of energy consumed per packet caused by the increase of packetlength. Larger packet size also increases the hop counts on the datapath. Fig.12.4a shows the combined effects of these factors. The values are normalizedto the measurement of 16Byte. As packet size increases, energy consumptionon the interconnect network will increase. The total energy dissipated on the MPSoC is shown in Fig. 12.4b. It clearlydecreases as packet size increases. However, when the packets are too large, asin the case of 256Byte in the figure, the total MPSoC energy will increase. Thisis because when the packet is too large, the increase of interconnect networkenergy will outgrow the decrease of energy on cache and memories.
  • 225 Core Shared memory Core−ports memory words North North South South East East counter West West Local Outputs Inputs Config. Line write operation read operation Crossbar Frequency Decoder Controller Voltage North to South & East PC Example Communication producer P1 P2 consumer Pattern Instruction Memory (a) ASOC Core Interface (b) Producer-consumer pair Figure 12.5. Communication-based power management12.7 SYSTEM AND APPLICATION LAYERS In the context of highly integrated on-chip multi-processors, lowering supplyvoltage of the cores reduces power quadratically but also results in a perfor-mance degradation which can be tolerated only if it does not impact performancebeyond a critical, application-dependent threshold. Given the key role playedby on-chip communication with respect to MPSoC performance, the conceptof communication-based power management (CBPM) has been introduced. Itconsists of integrating the system-level power management functionality intothe communication architecture, which binds the system components together,thus eliminating the need for separate power management entities. Second, dueto its connectivity, the communication architecture can gather information (suchas the execution states of system components) required to make power man-agement decisions. Finally, since the communication architecture schedulesinter-component communications, it can control the timing of a component’spower modes, thus regulating the component’s (and therefore the system’s)power profile. Multiple implementations of this concept are feasible [38, 6], and two rele-vant examples will be hereafter described. The first one is represented by theAdaptive System-on-Chip (ASOC) illustrated in [6], which has been used tobuild a backbone for power-aware signal processing cores [39]. ASOC abilityto provide dynamic voltage and frequency scaling is due to the architectureof the network interface of the cores, illustrated in Figure 12.5(a) [39]. Thecore interface uses a synchronized global communication schedule to managecommunications through each tile. The instruction memory holds a list of thecommunication patterns required at run-time. A program counter (PC) fetchesthese patterns in succession and a decoder converts them into switch settings for
  • 226a crossbar, that routes data between the local core and neighboring tiles (North,East, South or West). Moreover, at each core, frequency and voltage are au-tomatically adjusted. A subsystem uses up/down counters to track the datatransfer rate between core and interconnect. Blocked or unsuccessful transferscause the count to increase, while successful transfers decrease the value. Ifthe core input port is blocked consecutively, the core is running too slowly withrespect to its predecessors. If the core output port is consecutively blocked, thecore is running too quickly for its successors. In either case, these counters sendtrigger signals to the core configuration unit to increase or decrease the coreclock. The new frequency setting automatically selects a new supply voltagevalue. A similar approach can be applied to pipelined signal processing applica-tions, wherein a sequence of computation stages exchange the results of theirprocessing in a pipelined fashion. From an hardware viewpoint, the systemmight consist of cascaded producer-consumer pairs communicating by meansof a shared memory. If producer and consumer are not well synchronized,energy-inefficient synchronization mechanisms could be triggered. For in-stance, if the consumer expects input data to process while the producer is notready to output the result of its computation yet, the consumer keeps pollingon a semaphore until its input data is available in the shared memory. Thismechanism wastes significant amounts of energy, and should be avoided asmuch as possible. A solution to keep producer-consumer pairs synchronizedis reported in Figure 12.5(b). Shared memory can be abstracted as a queueand a memory access counter keeps track of the queue level. When a lowerthreshold is crossed, it means that the producer is too slow or the consumertoo fast, and frequency/voltage scaling can be applied for balancing data pro-duction or consumption rate. The opposite holds when an upper threshold iscrossed. Counter monitoring might be carried out by a proper power man-agement hardware module connected to the bus, with the ability to program(through a register) the clock frequency generator of the cores. This couldbe done continuously or periodically at discrete times, in order to amortizethe frequency switching cost. Worst-case power savings with respect to staticfrequency selection (power-optimized for a particular application) amounts to12%.12.8 APPLICATION-SPECIFIC NETWORKS-ON-CHIP Customizing MPSoC architectures and tailoring them to a specific appli-cation domain is a very promising approach to system energy minimization.It takes its steps from the optimization techniques used in some SoC designmethodologies, that explicitly target applications from a specific domain: their
  • 227main features (control flow, data organization and type of processing) are eval-uated and exploited for a power-aware architecture customization [8]. For customized NoCs to be successful, however, developers must select theappropriate domain-specific architecture and map the system’s communicationrequirements onto it [7]. This is a non-trivial task, in that an optimized net-work instance has to be derived by analyzing the application communicationrequirements, and by comparing a number of alternative interconnect solutions. Moreover, reusing the components of a given NoC architecture across dif-ferent designs (and therefore network instances) becomes feasible provided thenetwork building blocks (network interfaces, switches, switch-to-switch links)are designed as soft macros. Some NoC architectures proposed in the literature were built around thisconcept [6, 4, 3, 2]. Only two of them are mentioned for the sake of brevity.Quality of Service NoC (QNoC) [3] is a NoC framework wherein QoS and costmodel for communications in SoCs are first defined, and related NoC archi-tecture and design process are then derived. SoC inter-module communicationtraffic is classified into 4 classes of service: Signaling (control signals), Real-Time, RD/WR (for short data access) and Block-Transfer (for large data bursts).By analyzing the communication traffic of the target SoC, QoS requirements(in terms of delay and throughput) for each of the four service classes are de-rived. A customized QNoC architecture is then created by modifying a genericnetwork architecture (two-dimensional planar mesh, fixed shortest-path multi-class wormhole switching). The customization process minimizes the networkcost (in area and power) while maintaining the required QoS and works asfollows: the SoC modules are placed so as to minimize spatial traffic density,unnecessary mesh links and switching nodes are removed, and bandwidth isallocated to the remaining links and switches according to their relative load sothat link utilization is balanced. Finally, Xpipes NoC architecture [2] is a library of highly parameterizablenetwork components which are design-time tunable and composable to get cus-tomized domain-specific architectures. Xpipes has been designed with high-performance in mind, and this has been achieved by means of deeply pipelinedswitches, pipelined links to decouple link throughput from link delay, virtualoutput buffering. The network interface implements OCP standard signalingand the look-up tables required by static routing algorithms. The network inher-ently provides best-effort services and targets multi-gigahertz heterogeneousMPSoCs, wherein irregular network topologies with links of uneven lengthmight be required. Next, an instructive case study about application-specific NoC instances andtheir potentials for energy savings is reported, leveraging the Xpipes synthesisflow.
  • 228 357 vu med 70 run au rast vld le dec 27 iquan idct 16 cpu 60 40 190 0.5 362 353 Arm 600 40 inv 362 362 acdc up sdram 250 idct scan pred samp sram1 sram2 ,etc stripe 49 0.5 mem 910 670 313 vop 300 500adsp up 173 313 pad rec 16 samp 32 bab risc vop mem 94 500 (a) MPEG4 core graph (b) VOPD core graph Figure 12.6. Core Graphs of Video Processing Applications12.8.1 Case study A core graph representation of the application is the input to Xpipes-basedsynthesis flow (called NetChip). The design and generation of a customizedNoC is achieved by means of two tools: SUNMAP, which performs the networktopology mapping and selection functions, and ×pipesCompiler, which per-forms the topology generation function. SUNMAP produces a mapping of coresonto various NoC topologies that are defined in a topology library. The map-pings are optimized for the chosen design objective (such as minimizing area,power or latency) and satisfy the design constraints (such as area or bandwidthconstraints). SUNMAP uses floorplanning information early in the mapping pro-cess to determine the area-power estimates of a mapping and to produce feasiblemappings (satisfying the design constraints). The tool supports various routingfunctions (dimension ordered, minimum-path, traffic splitting across minimum-paths, traffic splitting across all paths) and chooses the mapping onto the besttopology from the library of available ones. A design file describing the chosentopology is input to the ×pipesCompiler, which automatically generates theSystemC description of the network components (switches, links and networkinterfaces) and their interconnection with the cores. A custom hand-mappedtopology specification can also be accepted by the NoC synthesizer, and thenetwork components with the selected configuration can be generated accord-ingly. NetChip was applied to two different video processing applications: VideoObject Plane Decoder (VOPD - mapped onto 12 cores), MPEG4 decoder (14cores). These are high-end video-processing applications and the hardware-software partitioning of the applications is presented in [35, 36]. The coregraphs of these applications is presented in Figure 12.6. The maximum linkbandwidth for the NoCs is conservatively assumed to be 500 MB/s. The results of mapping VOPD onto various topologies are presented in Fig-ure 12.7. As seen from Figure 12.7(a), the butterfly topology (4-ary 2-fly) has theleast communication delay out of all topologies, the least number of switches,
  • 229 3 45 60 500 40 Sws Lnks AreaAvg 35 (mm2) 400 PowerNo 2 SWs 30 40 (mW)Hops & 25 300 Lnks 20 200 1 15 20 10 100 5 0 0 0 0 Msh Trs Hyp Cls Bfly Mesh Torus Hyp Cls Bfly Msh Trs Hyp Cls Bfly Msh Trs Hyp Cls Bfly (a) Avg hop delay (b) Resource Util (c) Design Area (d) Design Power Figure 12.7. Mapping Characteristics of VOPD run inverse run inverse 44 VLD length VLD length Cust scan scan decoder decoder 42 Mesh s1 s2 s1 AC/DC iQuant AC/DC 40 iDCT iDCT iQuant Predict Predict Avg Pack Lat (Cy) s2 s3 s2 38 S2 S1 up Stripe VOP Stripe up VOP 36 samp reconstr Mem Mem samp reconstr s2 s3 34 s2 S2 S3 S2 ARM VOP ARM 32 Padding VOP core Mem core Padding Mem 30 s1 s2 s1 28 s1 − 3x3 s1 − 3x3 s2 − 4x4 s2 − 3x2 s3 − 2x3 26 s3 − 5x5 1.6 1.8 2 2.2 2.4 2.6 − repeater − repeater BW (GB/s) (a) Mesh NoC (b) Appln Specific NoC (c) VOPD Avg Lat Figure 12.8. Mesh and Custom Mappings of Video Object Plane Decoderbut has more links when compared to mesh, torus or hypercube. The largepower savings achieved by the butterfly network (Figure 12.7(d)) is attributedto the fact that there are fewer switches and smaller number of hops for commu-nication. Moreover, all the switches are 4x4, while the direct topologies have5x5 switches. The average link length in the butterfly network (obtained fromfloorplanner) was observed to be longer than the link lengths (around 1.5×) ofdirect networks. However, as the link power dissipation is much lower than theswitch power dissipation, we get large power savings for the butterfly network.The smaller number of switches and smaller switch sizes also account for thelarge area savings achieved by the butterfly network. Thus, butterfly is the besttopology for VOPD. The performance gains for the butterfly over other topolo-gies may be surprising, but careful inspection shows that the butterfly networktrades-off path diversity for network switches with average hop delay. On thecontrary, the same kind of analysis shows that a mesh topology is more suitablefor MPEG4 than other topologies.12.8.2 Generating Custom Topologies For custom topologies, the mapping and generation phases of the tool canbe skipped and ×pipesCompiler can be directly invoked on the input design.A custom hand-tuned NoC for the VOPD is presented in Figure 12.8(b). In
  • 230the VOPD, about half the cores communicate to more than a single core. Thismotivates the configuration of this custom NoC, having less than half the numberof switches than the mesh NoC. NetChip area and power reports relative to thecustom NoC were automatically obtained. Significant area (5x) and powerimprovements (2x) were noticed with the custom NoC as fewer, smaller sizeswitches are used with respect to the mesh network. SystemC simulation of the NoC models allowed to assess their performance.The variation of average packet latency (for 64B packets, 32 bit flits and 7 cycleswitch delay) with link bandwidth is showed in Figure 12.8(c). Application-specific NoC has lower packet latency as the average number of switch and linktraversals is lower. Moreover, the latency increases more rapidly for the meshNoC with decrease in bandwidth. With the custom NoC, an average of 25%savings in latency (measured at the minimum plotted BW value) is achieved.12.9 CONCLUSIONS This chapter focuses on low power design techniques for NoC-based gi-gascale MPSoCs. Several open problems were described at various layers ofthe communication stack, and the basic strategies to effectively tackle themwere sketched. Finally, the large potentials for energy savings provided by theimplementation of customized, domain-specific NoCs have been discussed.
  • 231References [1] P. Aldworth, “System-on-a-Chip Bus Architecture for Embedded Applications,” IEEE International Conference on Computer Design, pp. 297-298, 1999. [2] “×pipes: a Latency Insensitive Parameterized Network-on-chip A rchitecture For Multi-Processor SoCs”, M.Dall’Osso, G.Biccari, L.Giovannini, D.Bertozzi, L.Benini, Int. Conf. on Computer Design, pp.536-541, October 2003. [3] E.Bolotin, I.Cidon, R.Ginosar, A.Kolodny, "QNoC: QoS architecture and design process for Network on Chip", Journal on Systems Architecture, Special Issue on Networks on Chip, December 2003. [4] I.Saastamoinen, D.S.Tortosa, J.Nurmi, "Interconnect IP Node for Future System-on-Chip Designs", IEEE Int. Work. on Electronic Design, Test and Applications, pp.116-120, January 2002. [5] C.T. Hsieh, M. Pedram, ”Architectural Energy Optimization by Bus Splitting,” IEEE Trans. CAD, Vol.21, issue 4, pp.408-414, April 2002. [6] J.Liang, S.Swaminathan, R.Tessier,”aSOC: A Scalable, Single-Chip Communication Architecture,” IEEE Int. Conf. on Parallel Architectures and Compilation Techniques, pp.37-46, October 2000. [7] S.Murali, G.De Micheli, "Bandwidth-Constrained Mapping of Cores onto NoC Architectures", De- sign Automation and Testing in Europe, 2004, pp.20896-20901. [8] L.Bisdounis, C.Dre, S.Blionas, D.Metafas, A.Tatsaki, F.Ieromninon, E.Macii, P.Rouzet, R.Zafalon, L.Benini "Low-Power System-on-Chip Architecture for Wireless LANs," IEE Proc.-Comput. Digit. Tech., Vol.151, no1, January 2004. [9] K. Lee, S.J. Lee, S.E. Kim, H.M. Choi, D. Kim, S. Kim, M.W. Lee, H.J. Yoo, "A 51mW 1.6GHz On-Chip Network for Low Power Heterogeneous SoC Platform", IEEE Int.Solid-State Circuits Con- ference, pp.1-3, 2004.[10] S.J. Lee et al., "An 800MHz Star-Connected On-Chip Network for Application to Systems on a Chip ", IEEE Int.Solid-State Circuits Conference, pp.468-469, February 2003.[11] W. Bainbridge, S. Furber, “Delay insensitive system-on-chip interconnect using 1-of-4 data encod- ing,” IEEE International Symposium on Asynchronous Circuits and Systems, pp. 118-126, 2001.[12] D. Bertozzi, L. Benini and G. De Micheli, “Low-Power Error-Resilient Encoding for On-chip Data Busses,” DATE, International Conference on Design and Test Europe Paris, 2000, pp. 102-109.[13] Dally, W.; Towles, B.; “Route Packets, Not Wires: On-Chip Interconnection Networks” 38th Design Automation Conference, 2001. Proceedings[14] B. Cordan, “An efficient bus architecture for system-on-chip design,” IEEE Custom Integrated Cir- cuits Conference, pp. 623–626, 1999.[15] Dally, W.J; Aoki, H. “Deadlock -free adaptive routing in multicomputer networks using virtual channels” IEEE Trans. on Parallel and Distributed Systems, April 1993[16] W. Dally and J. Poulton, Digital Systems Engineering, Cambridge University Press, 1998.[17] J. Duato, S. Yalamanchili, L. Ni, Interconnection Networks: an Engineering Approach. IEEE Com- puter Society Press, 1997.[18] R. Hegde, N. Shanbhag, “Toward Achieving Energy Efficiency in Presence of Deep Submicron Noise,” IEEE Transactions on VLSI Systems, pp. 379–391, vol. 8, no. 4, August 2000.[19] R. Hegde, N. Shanbhag, “Toward achieving energy efficiency in presence of deep submicron noise,” IEEE Transactions on VLSI Systems, pp. 379–391, vol. 8, no. 4, August 2000.[20] R. Ho, K. Mai, M. Horowitz, “The Future of wires,” Proceedings of the IEEE, January 2001.[21] Karim, F.; Nguyen, A.; Dey, S. “On-chip Communication Architecture for OC-768 Network Proces- sors” 38th Design Automation Conference, 2001. Proceedings[22] E. Nilsson “Design and Implementation of a Hot-Potato Switch in a Network on Chip” Master of Science Thesis, LECS, Royal Institute of Technology[23] Li Shang and Li-Shiuan Peh and Niraj K. Jha, “Dynamic Voltage Scaling with Links for Power Optimization of Interconnection Networks,” HPCA - Proceedings of the International Symposium on High Performance Computer Architecture, Anaheim, February 2003, pp. 91-102.[24] Singh, J. P.; Weber, W.; Gupta, A,; “SPLASH: Stanford Parallel Applications for Shared-Memory” Computer Architecture News, vol. 20, no. 1[25] J. Walrand, P. Varaiya, High-Performance Communication Networks. Morgan Kaufman, 2000.
  • 232[26] F. Worm, P. Ienne, P. Thiran and G. De Micheli, “ An Adaptive Low-power Transmission Scheme for On-chip Networks,” ISSS, Proceedings of the International Symposium on System Synthesis, Kyoto, October 2002, pp. 92-100.[27] R. Yoshimura, T. Koat, S. Hatanaka, T. Matsuoka, K. Taniguchi, “DS-CDMA wired bus with simple interconnection topology for parallel processing system LSIs,” IEEE Solid-State Circuits Conference, pp. 371-371, Jan. 2000.[28] Ye, T. T.; Benini, L.; De Micheli, G.; “Packetized On-Chip Interconnect Communication Analysis for MPSoC” Design Automation and Test in Europe, DATE 2003 Proceedings[29] H. Zhang, V. George, J. Rabaey, “Low-swing on-chip signaling techniques: effectiveness and robust- ness,” IEEE Transactions on VLSI Systems, vol. 8, no. 3, pp. 264-272, June 2000.[30] H. Zhang, M. Wan, V. George, J. Rabaey, “Interconnect architecture exploration for low-energy configurable single-chip DSPs,” IEEE Computer Society Workshop on VLSI, pp. 2-8, 1999.[31] C.H. Zeferino, A.A. Susin, "SoCIN: A Parametric and Scalable Network-on-Chip," Symposium on Integrated Circuits and Systems Design SBCCI’03, pp. 169-174, September 2003.[32] F. Poletti, D. Bertozzi,L. Benini,A. Bogliolo, "Performance Analysis of Arbitration Policies for SoC Communication Architectures," Journal on Design Automation for Embedded Systems, Kluwer, pp. 189-210, 2003.[33] IBM CoreConnect bus architecture, ”http://www-3.ibm.com/chips/products/coreconnect”[34] AMBA Multi-Layer AHB and AHB-Lite, ”http://www.arm.com/products/solutions/AMBAAHBandLite.html”[35] E.B.Van der Tol, E.G.T.Jaspers,"Mapping of MPEG-4 Decoding on a Flexible Architecture Platform", SPIE 2002, pp. 1-13, Jan, 2002.[36] E.G.T.Jaspers, et al.,"Chip-set for Video Display of Multimedia Information", IEEE Trans. on Con- sumer Electronics, Vol.4 5, No. 3, pp. 707-716, Aug, 1999.[37] R.H. Havemann, J.A. Hutchby, ”High-Performance Interconnects: An Integration Overview”, Pro- ceedings of the IEEE, Vol.89, no5, pp.586-601,May 2001[38] K. Lahiri, A. Raghunathan, S. Dey ,"Communication Architecture Based Power Management for Battery Efficient System Design", Proc. ACM/IEEE DAC, pp.691-696, 2002.[39] A. Laffely, J. Liang, R. Tessier, W. Burleson ,"Adaptive System on Chip (aSoC): a Backbone for Power-Aware Signal Processing Cores", Int. Conf. on Image Processing, pp.105-108 (III), 2003.[40] V. Raghunathan, M.B. Srivastava, R.K. Gupta ,"A Survey of Techniques for Energy Efficient On-Chip Communication", DAC 2003, pp.900-905, June 2003.
  • 233Chapter 13SYSTEM LEVEL POWER MODELING ANDSIMULATION OF HIGH-END INDUSTRIALNETWORK-ON-CHIPAndrea Bona , Vittorio Zaccaria and Roberto ZafalonSTMicroelectronicsAbstract Today’s System on Chip (SoC) technology can achieve unprecedented computing speed that is shifting the IC design bottleneck from computation capacity to communication bandwidth and flexibility. This chapter presents an innovative methodology for automatically generating the energy models of a versatile and parametric on-chip communication IP (STBus). Eventually, those models are linked to a standard SystemC simulator, running at BCA and TLM abstraction level. To make the system power simulation fast and effective, we enhanced the STBus class library with a new set of power profiling features (“Power API”), allowing performing power analysis either statically (i.e.: total avg. power) or at simulation runtime (i.e.: dynamic profiling). In addition to random patterns, our methodology has been extensively benchmarked with the high-level SystemC simulation of a real world multi-processor platform (MP- ARM). It consists of four ARM7TDMI processors accessing a number of peripheral targets (including several banks of SRAMs, Interrupt’s slaves and ROMs) through the STBus communication infrastructure. The power analysis of the benchmark platform proves to be effective and highly correlated, with an average error of 2% and a RMS of 0.015 mW vs. the reference (i.e. gate level) power figures. The chapter ends presenting a new and effective methodology to minimize the Design of Experiments (DoE) needed to characterize the above power models. The experimental figures show that our DoE optimization techniques are able to trade off power modeling approximation with characterization cost, leading to a 60% average reduction of the sampling space, with 20% of maximum error.Keywords: Network-on-Chip power analysis, communication based low power design, system-level energy optimization.
  • 234 13.1 INTRODUCTION Embedded computing systems are on the way to provide a number of new services that will arguably become common practice in the next few years. The most important of these are (i) multimedia (audio/video streaming) capabilities in personal communicators, (ii) huge computing power (especially from clusters of processors) and storage size, (iii) high rate accessibility from mobile terminals. Today’s System on Chip (SoC) technology can achieve unprecedented computing speed that is shifting the IC design bottleneck from computation capacity to communication bandwidth and flexibility. SoC’s designers need to leverage on pre-validated components and IPs such as processor cores, controllers and memory arrays. Design methodology will further support IP re-use in a plug-and-play fashion, including buses and hierarchical interconnection infrastructures. SoCs will have to provide a functionally correct, reliable operation under data uncertainty and noisy signaling. The on-chip physical interconnection will be a limiting factor for both performance and energy consumption, also because the demand for component interfaces will steadily scale-up in size and complexity. In this chapter, we will present a thorough methodology for automatically building the energy model of a Network-on-Chip (NoC) IP at the BCA/Transaction level, in order to allow power profiling of an entire platform since the very early stages of the system design, often when only a software model of the system does exist. The chapter is organized as follows: Section 13.2 introduces a short background on Network-on-Chip. Section 13.3 illustrates the STBus versatile interconnect IP as an industrial example of NoC infrastructure. Section 13.4 introduces the overall NoC power characterization and estimation framework while Section 13.5 goes into details about our NoC’s energy model. Section 13.6 presents the Design of Experiment policy and Section 13.7 reports a significant set of figures about the model validation and the experimental results, including a real-world platform simulation case. Eventually, Section 13.8 presents a new and effective methodology to minimize the Design of Experiments (DoE). 13.2 BACKGROUND Although the main concepts and the terminology of Network-on-Chip design has been introduced quite recently [1][2][3], both the industrial and research communities have been starting to realize the strategic importance of shifting the design paradigm of high-end digital IC from a deterministic,
  • 235wire-based interconnection of individual blocks and IPs, to a thoroughcommunication-based design methodology [4][7][9], aiming to face withdata packets and non-deterministic communication protocols in nextgeneration’s SoCs. With the advent of 90nm and 65nm CMOS technology, the challenges tofix the Network-on-Chip (NoC) issue “by design”, will need: – To provide a functionally-correct, reliable operation of theinterconnected components by exploiting appropriate network infrastructureand protocols, i.e. interconnections to be intended as “on chip micro-network” [5][6][7], which is an adaptation of the OSI protocol stack [18]. – To achieve a fluid “flexibility vs. energy-efficiency” systemexploration, allowing an effective network centric power management[8][11][12]. Unlike computation energy in fact, the energy for globalcommunication does not scale down with technology shrinking [3][4]. Thismakes energy more and more dominant in communications. Reaching those goals will be crucial to the whole semiconductor industryin the next future, in order to face with the escalating range of signalintegrity and physical wiring issues, who are making the target IC reliabilityharder and exponentially expensive to achieve. As of today, there is a limitedavailability of tools able to consistently support this emerging designmethodology. Indeed, some high level models for functional/performancesystem simulations (i.e. Bus Cycle Accurate and Transaction) are smoothlycoming up [13] across the design community. However, power predictabilityof NoCs still remains an open issue. Although NoC’s power estimation has been partially addressed in [10],its low level modeling (i.e. gate and device level) and the extremely slowsimulation (i.e. 1000 cycle/s) makes it definitely unsuitable to face with anysystem level SW/HW exploration task, which might easily need forsimulation speeds larger than 100 Kcycle/s.13.3 ON-CHIP NETWORK: STBUS INTERCONNECT STBus is versatile, high performances interconnect IP, allowing tospecify the communication infrastructure in terms of protocol, interface andparametric architectures [14][15]. It comes with an automated environment(STBus generation kit) suitable to support the whole design flow, startingfrom the system-level parametric network specification, all the way down tothe mapped design and global interconnect floor-plan [16]. The protocolmodes supported by STBus are compliant with VSIA standard [19]. In fact,they can scale up from Peripheral, to Basic and to Advanced mode,conventionally named Type-1, Type-2 and Type-3, respectively. In thiswork, we focus on the last 2 protocols (i.e. Type-2 and Type-3) since they
  • 236 better fit with the high demanding communication resources required by modern SoCs. More specifically, Type-2 supports pipelined split transactions, where each transaction is composed by a pair of send and receive packets (packet: a sequence of atomic messages called cells). On top of the above features, Type-3 allows to manage out-of-order packet delivery. The datapath’s width can range between 32, 64 and 128 bits. The STBus architecture builds upon the node module, configurable switch fabrics who can be instantiated multiple times to create a hierarchical interconnect structure. The topology of the switch fabric can be selected by choosing the number of resources dedicated to the request and the response packets; for example a shared bus interconnect has only 1 request and 1 response resources at a time, while a full cross-bar has as many request and response resources as the number of initiators and targets connected to the node. Eventually, type converter and size converter modules can be adopted to interface heterogeneous network domains working under different protocols (i.e. Type-1, 2 and 3) and/or different data-path widths. 13.4 ENABLING ENERGY EXPLORATION FOR NOC When dealing with multi-processors embedded systems, characterized by tens of masters and slaves connected through a complex communication infrastructure, energy estimation and optimization become of utmost importance. As a matter of fact, although more effective than traditional buses, NoCs are expected to make a relevant contribution to the area budget, due to the growing complexity of packet routing and transaction management policies affecting the interconnection’s control-path, and to the switch fabric in charge of supporting the high speed data packet delivery. Such a complexity has a cost in terms of energy consumption that should be traded-off with the performance benefits. Network structures achieving lower packet’s congestion (i.e. higher performance), are usually characterized by larger data-path complexity in terms of number of simultaneous routing resources available for packet broadcasting. For example, a shared bus communication node can be slower (i.e. higher congestion), yet less power consuming than a full crossbar switch-box, or, the slot-reservation arbitration policy may overcome the limitation of Time Division Multiple Access (TDMA) policy in case of asymmetric workloads in a multi-processors platform. These questions need to account for energy metric during the design exploration in order to find out the optimal platform configuration to meet the performance constraints at minimum energy. Exploration and optimization for SoC design are rapidly evolving towards the analysis of abstract description models that mimic the main operations of the system under analysis, including speed and power
  • 237behavior. According to the SystemC modeling scenario depicted in [13], theabstraction levels that can be used to model the function/power/performanceof a communication-based system are the Functional un-timed level, theTransaction level (TLM), the Bus Cycle Accurate level (BCA) and the PinAccurate – Cycle Accurate level (PA-CA). In short, while the Functionallevel does not give any insight on the timing figures of the system, theTransaction level only gives coarse time hints (e.g. total read/write timeslot), with no structural information on actual wires or pins. The BCA levelachieves cycle-accurate timing estimates, yet functionally accurate at theboundaries, while the PA-CA goes down to a clock cycle timing withstructural pin-accurate description, at the expense of a much slowersimulation. In this chapter we introduce a consistent methodology forautomatic energy model’s building to fit most of the above abstraction levels(i.e. Transaction, BCA, PA-CA), suitable to support the NoC’s powerestimation since the very early stages of the design exploration, when only aC/C++ model of the system is usually available. Eventually, the system simulation (developed in SystemC, in our case)will rely on high-level profiling statistics to figure out the energy cost, bymeans of an appropriate library of energy views and a dedicated proceduralinterface (API). In the following, we will explain how the STBus energymodels are based on a set of parametric, analytic equations that areindividually accessed by the simulator to compute the eventual energyfigures (either statically or at simulation runtime).13.4.1 Energy Characterization Flow The energy macro-model of the whole STBus interconnection ispartitioned into sub-components, corresponding to each micro-architecturalblock of the interconnection fabrics that are node, type-converter and size-converter. For sake of simplicity, in this chapter we will show the results ofthe node component. However, the same automatic flow is currently appliedto all of the components of STBus architecture. The proposed model relieson the bus utilization rate, i.e. the number of cells traveling across the bus, aswell as on the interconnection topology (i.e. the number of masters/targets),which need to be pre-characterized, once and for all, through an accurategate-level simulation for each target technology. The power characterization flow consists of 4 major steps depicted inFigure 13-1.
  • 238 Abstract CoreConsultant/Design Compiler Network Synthesis Topology STBus Generation Kit Gate-level Netlist Testbench Generation VCS/PowerCompiler Gate-level power est. Power & High-level stats Power Model Characterization Models DB Figure 13-1. STBus Power Characterization Flow As already mentioned in section 3, the STBus Generation Kit allows the designer to automatically synthesize a gate-level netlist starting from a system-level parametric network specification. This is done by inferring the corresponding RTL code and, then, synthesizing all the way down to the mapped design [16]. Thus, an extensive set of gate-level power simulations is launched within a Test-bench Generation suite, specifically tuned to fulfill the many requirements imposed by the STBus protocols and, at the same time, to sensitize the node under a wide range of traffic workloads. Actually, the test-benches can be configured in terms of average latency per master request and slave response and type of operations to be performed on the bus. The operations can be divided in two categories (Load and Store) as they can play with different operand sizes (from 1 to 32 bytes). The last step of the flow in Figure 13-1 is the Model Characterization, where each of the coefficients is computed to fit the high-level model (ref. to next section 5 for details). The final models (one for each component and target technology) are stored into a centralized Power Model Database. Sure enough, the choice of experiments, the length of each simulation and the test-benches adopted during the characterization campaign are crucial knobs to be optimized before running the characterization flow, by means of a suitable Design of Experiments (DoE: see section 6).
  • 23913.4.2 Hooking up the energy models into the system simulation The STBus Generation Kit supports the generation, among the others, ofthe SystemC model of each component, ready to be plugged into the targetSystemC simulation platform. The current release of the STBus GenerationKit is compliant with BCA SystemC v2.0 descriptions [13]. In evolution, thesupport for TLM is planned soon, according to the STBus roadmap. Theoverall SystemC power estimation flow is outlined in Figure 13-2. To makethe system simulation environment fast an effective, an ad-hoc API has beendeveloped (SystemC Power API), together with a consistent library offunctions allowing to enhance the basic SystemC capabilities with a powerprofiling feature, providing power analysis either statically (i.e.: total avg.power) or at simulation runtime (i.e.: dynamic profiling). The latter is doneby computing a moving average power on a given time window (e.g. tenclock cycles). Abstract Network STBus Generation Kit Topology SystemC Node Power Profile Energy Enhanced SystemC Node SystemC Power API Power Models DB Figure 13-2. Power Enhanced SystemC Simulation By deriving the SystemC node classes and hooking them up to thespecific SystemC Power API, we achieve the energy flow enhancement. As amatter of fact, energy-enhanced SystemC nodes provide an extremely fastprocedural interface to retrieve each set of model’s coefficients out from thepower model database as well as an effective power analysis during theSystemC simulation run.
  • 240 13.5 STBUS ENERGY MODEL In this section, we introduce the power model for a generic configuration n of a node. The configuration of an STBus node identifies a specific instance out from the design space S: S = {n | n =< i, t , rqr , rpr , p, C L , dps, Type >} (1) where i is the number of initiators, t is the number of targets, rqr is the number of request resources, rpr is the number of response resources, p is the type of arbitration policy (STBus has 7 arbitration policies), CL is the output pin capacitance (range: CLmin= 4 Standard Loads ; CLmax=1 pF), dps is the data-path size (range: 32, 64 and 128 bit) and Type is the protocol mode (Type-2 and 3, in this case). Based on an extensive experimental background, we recognize a fairly linear relationship between node energy and the rate of sent and received packet cells across all of the interconnection node’s ports. Such a behavior matches with a set of random configuration samples across the entire design space and it has been confirmed during the model validation phase (see section 7). The energy model for a generic configuration n of the STBus node is the following: E (n ) = P(n ) ⋅ C ⋅ Tclk (2) where P(n) is the average power consumption of the node during a simulation of C clock cycles, with a clock period of Tclk. The power consumption P(n) is a linear combination of three contributions, according to the following equation: rs r (3) P (n ) = B(n ) + Psent (n ) ⋅ + Prec (n ) ⋅ r C C where B(n) is the average base cost depending on the specific configuration n of the node, Psent(n) is the additive power cost due to cell sent from the masters to the slaves and rs is the total number of cells sent, Preq(n) is the power cost due to each packet cells received by the masters, rr is the total number of cells received by the masters and C is the number of clock cycles. In essence, the power model characterization consists in determining the value of the coefficients B(n), Psent(n) and Preq(n) for each
  • 241specific configuration n of the node. As formerly mentioned, this task isperformed by means of a polynomial regression over the set of experimentsgiven by DoE (see section 6). Although linear regression has beensuccessfully used to build the model’s coefficients so far, any higher ordermodels can be easily adopted should a better accuracy become an issue. Theexperimental setup is generated with the goal of properly stressing rs and rrover the whole range of variation. The total avg. switching activity comingout from the test-benches is kept at 0.5. As far as the interconnectioncapacitive load “CL“ is concerned, our model supports a linear interpolationbetween CLmin and CLmax in order to provide a quite accurate estimation of theswitching power under the actual instance’s output load. From a global viewpoint, the characterization campaign of STBus acrossthe whole design space may easily become a huge computing task. Thecomputational effort to power characterize STBus is similar or even largerthan the characterization of an industrial size ASIC library. The wholecomprehensive STBus design-space, in fact, would lead to more than3.4*105 individual configurations to be characterized (i.e. RTL synthesis +gate-level simulation + power measure). Such a number comes out from theproduct of all the possible combinations of the STBus design subspaces (i.e.8 initiators, 8 targets, 8 request and 8 response resources, 7 arbitrationpolicies, 2 load capacitances, 3 data path sizes and 2 types of protocols).Running an exhaustive characterization is far to be feasible in a reasonabletime, even by leveraging on distributed computers. We decided to adopt aresponse surface method approach to solve this problem. In this approach,only a selected set of configurations are synthesized and characterized,making the remaining set of coefficients derivable by accessing anappropriate set of models (either analytic or look-up table) obtained throughresponse surface methods. Although this approach may lead to someinaccuracy with the energy estimation process, the global accuracy can betaken well under control while allowing a remarkable saving incharacterization effort.13.6 OPTIMAL DESIGN OF EXPERIMENTS The fundamental theory on statistical design has been largelyconsolidated during the last twenty years or so, for a wide variety ofapplications [20]. In this context, the Design of Experiments is based on theconvergence analysis of some peculiar quality figures such as averagepower and average prediction error. Converging to the average powerfigure let us to identify the minimum length necessary for each simulation,by considering when the power consumption gets close to a steady value,given an arbitrary acceptance threshold (see the power–time curve of Figure
  • 242 13-3). On the other hand, the minimum number of experiments (i.e.: synthesis + simulations) needed to safely probe the design space and characterize the specific model, strongly depends on the target accuracy (i.e. max prediction error) as well as on the acceptable characterization effort. The regression analysis to fit the model’s coefficients is performed on the raw characterization data. Therefore, the QoR can be analytically measured by the prediction correlation coefficients (R and R2) and the Root Mean Square error (RMS). Eventually the minimum number of experiments is identified by considering both the RMS steady state and the absolute error over a set of significant benchmarks. 13.6.1 Convergence Analysis: Average Power The minimum simulations length necessary for the model characterization has to be identified through a convergence analysis. While the minimum simulations length of the testbench would not affect the actual power consumption, it is crucial to make sure that the circuit under analysis can always reach a steady state functional activity before measuring the avg. power consumption. To identify the correct simulation length, we minimized a cost-function that is a product of the simulation time and a measure of the derivative of the power consumption. The cost function is the following: ∆P(t ) C (t ) = t 2 ⋅ (4) P(t ) where t is the simulation time, P(t) is the power consumption measured at time t and P(t) is the difference between P(t) and P(t-1). Figure 13-3 shows the average behavior of the cost function for all the possible configurations of shared bus, Type-3, 32 bit width nodes. As can be seen, after 5000 ns the difference between power values does not pay for the increased simulation time. Thus, 5000 ns have been selected as the simulation length for all the characterization experiments.
  • 243 Accuracy - Tim e Cost Function 1300000 1100000 Cost function 900000 700000 500000 300000 0 2000 4000 6000 8000 10000 Sim ulation Tim e [ns] Figure 13-3. Avg. Power vs. Simulation-Time convergence analysis for a given STBus node’s configuration The derivative (i.e. differential ratio) has been sampled every 1000 nsand, then, normalized to the related power values in order to give apercentage variation.13.6.2 Convergence Analysis: Model Accuracy According to previous section 5, the model’s coefficients are resultingfrom the polynomial regression over a given set of experiments. Thoseexperiments are generated according to the DoE’s policy, by stochasticallychanging the number of data packets sent/transmitted across the bus and theoperation modes. The goal is to find out the minimum number ofexperiments necessary to meet the required accuracy. Given a set ofrepresentative STBus nodes, we perform their characterization with anincreasing number i of experiments. For each set of i calibrationexperiments, the Root Mean Square error (RMS) is evaluated. Figure 13-4shows that, for i>160, the RMS for all the configurations of the design spacegets close to the respective asymptotic values, with a maximum valuebounded to 0.01 mW. The minimum number of experiments to proceed withthe characterization of the STBus nodes has been defined accordingly.
  • 244 RMS Error Steady State 0.012 0.01 RMS [mW] 0.008 0.006 0.004 0.002 0 0 50 100 150 200 n. of experiments 8x8 Shared Bus - Type 3 - 32 bit 8x1 Shared Bus - Type 3 - 32 bit 1x8 Shared Bus - Type 3 - 32 bit 1x1 Shared Bus - Type 3 - 32 bit Figure 13-4. Power model’s RMS Error vs. Number of calibration experiments, under four different initiators/targets configurations 13.7 STBUS POWER MODEL VALIDATION AND EXPERIMENTAL RESULTS We present hereafter the results obtained from the validation phase of the proposed power macro modeling. In addition to the validation carried out by applying an extensive set of synthetic test-benches, we extended the test by running a realistic application, featuring mission-mode SystemC simulations of a multi-processors platform. All the characterization and experimental results presented in this chapter are targeted to STMicroelectronics HCMOS9 ASIC library, featuring 8 metal layers and 0.13 µm MOS channel length, operating at 1.2V nominal supply voltage. 13.7.1 Random pattern validation We carried out a synthetic validation by applying a uniform set of stochastically generated Verilog test-benches, similar to those used during the calibration phase (section 6.2). In Figure 13-5 we illustrate the scatter plot between the model estimation and the reference power measurement (coming from detailed gate-level power analysis). The average error is 1% with a correlation R of 96%.
  • 245 Measured Power Estimated Power 0Figure 13-5. Scatter plot of Measured vs. Estimated power consumption, for a set of synthetic benchmarks13.7.2 Mission mode validation through SystemC co-simulation To extensively validate our methodology into a real world simulationplatform, we decided to assess the robustness of the power model bycorrelating the power estimation coming from a high-level SystemCsimulation with respect to the gate-level power measure of the synthesizedSTBus node subject to the input stream generated at runtime by SystemC.The multi-processor platform is outlined in Figure 13-6. The architectureconsists of four ARM7TDMI processors accessing a number of targets(including several banks of SRAMs, Interrupt’s slaves and ROMs) throughthe STBus communication infrastructure, configured as a 4 initiators, 3targets, Type-3, shared bus, 32 bit, fixed priority request arbitration policy,dynamic priority response arbitration policy.
  • 246 SystemC SystemC SystemC SystemC Wrapper Wrapper Wrapper Wrapper RTEMS OS RTEMS OS RTEMS OS RTEMS OS ARM 7 TDMI ARM 7 TDMI … ARM 7 TDMI ARM 7 TDMI ISS ISS ISS ISS 8x8 shared bus – 32 bit – Type 3 STBUS Interconnect ROM … Shared RAM Private #1 RAM … Private #4 RAM Figure 13-6. Multi-processor platform including four ARM7TDMI processors connected through STBus Indeed, a remarkable amount of SW layers are intended to be executed on top of this HW platform, including a distributed real-time operating system who runs on each individual processor (RTEMS), and a class of multi-tasking DSP applications, featuring intensive integer matrix computations. Bus Traffic Rate Sent Cells Received Cells 1 0.9 0.8 0.7 0.6 Rate 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 60 70 80 90 Time - 10 KCycles Figure 13-7. Data packet rate monitored across the STBus on the target multiprocessor platform As far as the simulation framework is concerned, each processor’s ISS has been encapsulated with a SystemC wrapper, in charge of managing the interface protocol with the STBus communication node. The whole SW benchmark has total execution duration of 1 Million clock cycles, including the RT-OS booth strap (the initial 200 Kcycles) and the execution of the
  • 247DSP application SW. In Figure 13-7, the data cell’s statistics (i.e. rate ofcells sent/received per time unit) across the STBus is reported. The overallSystemC/Verilog co-simulation flow is depicted in Figure 13-8. Initiator 1 Co-simulation Interchange … Format File. ST proprietary SystemC Initiator n # Cosimulation interchange file - Generated for SystemC shared_2x1ICN.Node_1 # Cosimulation interchange file - Generated for # This file has been automatically generated by shared_2x1ICN.Node_1 a SystemCfile has been automatically generated by # This simualtion of STbus # STBus is asimualtion of STbus a SystemC propietary IP of STMicroelectronics Target 1 STBus … # STBus is a propietary IP of STMicroelectronics .dt init_0_data .vector 32 0x0 0 Node init_0_data .dt init_0_data .dt init_0_add init_0_data init_0_add .dt init_0_add .vector 32 0x0 .vector 32 0x1 0 .vector 32 0x1 0 0 Target m .dt init_0_req init_0_add init_0_req .dt init_0_req .dt init_0_eop init_0_req .vector 1 0x2 .vector 1 0x2 .vector 1 0x3 0 0 0 init_0_eop .dt init_0_eop .vector 1 0x3 0 .dt init_0_be init_0_eop .vector 8 0x4 0 Cosimulation init_0_be .dt init_0_be .dt init_0_opc init_0_be init_0_opc .dt init_0_opc .vector 8 0x4 .vector 8 0x5 .vector 8 0x5 0 0 0 Enhanced Node init_0_opc AST - Power stats Macro Model ∆ Verilog Verilog Co-simulation Synthesized Testbench STBus Node Generator Power saif Compiler VCS Figure 13-8. SystemC/Verilog co-simulation flow During the SystemC simulation, initiators and targets generate a trace of“mission mode” transactions, monitored through a specific feature of theSTBus node. In fact, the node has been enhanced in order to gather the fullsignals stream out from the SystemC simulation session. The eventual tracefile carries comprehensive print-on-change information, sampled on a clockcycle basis. The co-simulation file is then applied to drive the gate-levelVerilog simulation (VCS [16]) and, then, feeding the detailed power analysisof the mapped netlist (PowerCompiler [16]). In Figure 13-9 we compare the power predicted by SystemC whenrunning the system simulation (“Power estimated”) vs. the reference powermeasured by Power Compiler at gate-level (“Power measured”). Pleasenotice that absolute power numbers are hidden for technologyconfidentiality. The system level estimation proves to be highly correlated to thereference power figure, with an average error of 2% and a RMS of 0.015mW.
  • 248 Cosimulation - Power Report Power Measured [mW] Power Estimated [mW] Power [mW] 0 1 2 3 4 5 7 8 9 10 100K Clock Cycles Figure 13-9. Estimated vs. Measured average power in STBus 13.8 LOW EFFORT, HIGH ACCURACY POWER MACRO MODELING Due to the huge domain size, the optimization of the Design of Experiments (DoE) to be adopted when characterizing the power macro- model is a key methodology issue to eventually ensure the task’s feasibility. We decided to adopt a response surface method approach to allow this problem to be manageable and actually solvable through an automatic tool flow. Indeed, although this methodology has been developed to cope with STBus, the described solution can be easily applied to a number of generic parametric IP as well as to third party’s on chip Bus architectures. 13.8.1 Surface reconstruction methods Let us consider Equation 1 and Equation 3 (Section 5). As can be seen, the coefficients are functions over the dominium S, which can be considered the cartesian product of two subspaces: S= G×X (5) G = { g | g = <i, t > } (6) X = { x | x = < rqr, rpr, p, CL, dps, Type> } (7)
  • 249 The coefficients B, Psent and Prec can be seen as a function of twovariables: f(n) = f(g,x) (8) The variable g belongs to a discretized space called ‘grid’, over the set ofpossible pairs <initiator,target>, while variable x represents the remainingparameters, according to Equation (7). By considering x fixed, we have that each coeffiecient is described by asurface over the set G. Experimentally, the surface shows to have a verysmooth behavior; as an example Figure 13-10 shows a representativebehavior of the coefficient B. Energy / Sent Cell Power Initiators Targets Figure 13-10. An example surface for the base cost The problem of fast characterization of the entire design space can bethought as reconstructing a surface by directly characterizing only a smallsubset of points Gs ⊂ G. The given methodology must assure that for anincreasing ratio z=|Gs|/|G|, (i.e., the characterization effort of the powermodel library), the approximation error decreases. Ideally, for z=1, the errorshould be 0. Our approach, can be decomposed in three fundamental steps:1. Choice of a representative set Xs ⊂ X to be used as a training set for the evaluation of the different reconstruction methods.
  • 250 2. Automatic optimization of the surface reconstruction methods over the space G × Xs. The output of this stage will be a combination of algorithm and sub-grid Gs suitable to minimize fitting error and characterization effort. 3. Perform the actual characterization of Gs× X. The considered surface reconstruction methods are twofold. • Regression–based algorithms: Analytic approximation of a surface, usually a polynomial expression, in which each of the coefficients is fitted to minimize an overall square error. The procedure to fit these parameters is often called Least Square Approximation. Three regression methods have been evaluated: linear, quadratic and cubic. • Interpolation–based algorithms: Surface approximation by using multiple piecewise polynomials or more generic functions. Being based on look-up tables, the response surface always fits with all of the training set points. The following interpolation algorithms have been analyzed: cubic, bicubic, bilinear and linear grid data. Algorithms belonging to the “spline” interpolation class have been experimentally rejected due to a worse accuracy. As far as the sub-grid constraints are concerned, the regression based methods do not enforce any limitation on the sub-grid topology Gs , while interpolation methods have several topology constraints, for which the reader is referred to the specific literature [20]. Choice of a reference training set Xs ⊂ X The training set Xs is composed by a random selection within X such that each value of the parameters rqr, rpr, p, CL, dps, Type (see eq. 5) is evaluated at least once. In this way it is possible to uniformly visit the characterization space. More specifically in our design case, considering 8 request and 8 response resources, 7 arbitration policies, 2 load capacitances, 3 data path sizes and 2 protocols, our method allows to reduce the training set from 3.4*105 down to 30 configurations, without any significative degradation in accuracy. Surface Reconstruction Algorithm Benchmarking Each of the afore mentioned algorithms has been benchmarked by varying the ratio z=|Gs|/|G| and by sampling the design space G in order to
  • 251find an optimal solution Gs. The search for the optimal solution is driven bythe following cost function: C(Gs) = σ (Gs)η(Gs) (9) Where σ(Gs) is a guess of the characterization effort and η(Gs) is themaximum error between the predicted coefficients and the actualcoefficients. Then, for each value of z=|Gs|/|G|, we find out the optimal reconstructionalgorithm and the corresponding grid Gs to be adopted for the finalcharacterization campaign.13.8.2 Experimental results In this paragraph we show the experimental evaluations supporting theproposed methodology. The experimental flow aims at selecting the optimalgrid Gs together with the best reconstruction algorithm for a given grid size|Gs|. The analysis is performed on a subset of the entire design spaceconfigurations (Xs ⊂ X), as explained in section 8.1.1. For each algorithm and grid size |Gs|, an iterative random search isperformed to find the best grid, by optimizing the cost function specified inEquation 9. While driving the grid-optimization with the full cost functionC(Gs) (see eq. 9), we will use the maximum error η(Gs) to discriminatebetween candidate reconstruction methods, given a fixed grid-size. Combined Analysis To compare the various methods we are going to focus our attention onthe best surface reconstruction method for each category (regression andinterpolation). As far as the regression is concerned, the quadratic algorithmshows to be the most promising for small grid-sizes, even better than theinterpolation-based methods. On the other side, regarding interpolation, thecubic grid data technique is the most accurate in terms of maximum error.Accordingly, Figure 13-11 shows the best pair <σ, η> (see eq. 9) for eachgrid size |Gs|, with respect to Quadratic Regression (shortened as‘Regression’) and Cubic Grid Data Interpolation (shortened as‘Interpolation’). Moreover, the figure shows the Pareto curve highlightingpoints that are not dominated in both directions by any other point. Thiscurve is useful to screen out the solutions that optimize both the merit figuresand, at the same time, meet specific constraints. For example, a quadraticregression method should be used if the characterization effort is adominating constraint. In fact, in this case we have only 10% of the original
  • 252 characterization effort at the expense of a 33% maximum error. For medium- high characterization effort, the Cubic Grid Data interpolation has shown to reduce the maximum error more than the other methods. It leads to a maximum error of about 18% at the expense of 62% of the original characterization effort. Accuracy/Effort Trade-off 100% 90% 80% Characterization effort 70% 60% Pareto Curve 50% 40% 30% 20% 10% Interpolation Regression 0% 0% 10% 20% 30% 40% Maximum Error Figure 13-11. Characterization effort vs. Maximum error Trade-off 13.9 CONCLUSIONS An innovative methodology for automatically generating the energy models of a versatile and parametric on-chip communication infrastructure (STBus) has been presented in this chapter. The methodology aggressively targets correlated power estimation with efficient SystemC simulation, running at BCA and TLM abstraction level. Among other synthetic benchmarks, the NoC’s power models validation has been extensively addressing the high-level SystemC simulation of a real world multi- processor platform (MP-ARM), which includes four ARM7TDMI processors accessing a number of peripheral targets (including several banks of SRAMs, Interrupt’s slaves and ROMs) through the STBus communication infrastructure. All the characterization and experimental
  • 253results presented in this chapter are targeted to STMicroelectronicsHCMOS9 ASIC library, featuring 8 metal layers and 0.13 µm MOS channellength, operating at 1.2V nominal supply voltage. The synthetic validation between the model estimation and the referencepower figures (i.e. gate-level power measure of the synthesized NoC) showsan average error of 1% and correlation R of 96%. The power analysis ofthe MP-ARM benchmark proves to be highly effective and correlated, withan average error of 2% and a RMS of 0.015 mW vs. the reference power.Eventually, we presented a new and effective methodology to minimize theDesign of Experiments (DoE) needed to characterize a set of innovativeenergy macro models of Network-on-Chip architectures. We have shownthat, by properly combining regression (polynomial) and interpolation (table-based) techniques, this methodology aims to reduce the overwhelmingcomplexity of sampling the huge, non-linear design space involved with thesystem operations of an high-end parametric on-chip-network module. Theexperimental figures show that our DoE optimization techniques are able totrade off power modeling approximation with model building cost, leadingto an average 62% reduction of the sampling space with a maximum error of18%. Acknowledgements The authors are grateful to dr. C.Pistritto and his CMG/OCCS team inCatania, for their valuable and synergic support to achieve the leapenhancement of making the STBus Design Flow truly "power aware". References[1]J.Duato, S.Yalamanchili, L. Ni, “Interconnection Networks: an Engineering Approach”, IEEE Computer Society Press, 1997.[2] K. Lahiri, S.Dey et al.,”Efficient Exploration of the SOC Communication Architecture Design Space”, Proc. of ICCAD-2000, Nov. 2000, S.Jose`, USA.[3] W. Dally, B. Toles, “Route Packets, not Wires: On-Chip Interconnection Network”, Proceedings of 38th DAC 2001, June 2001, Las Vegas, USA.[4] A. Sangiovanni Vincentelli, J. Rabaey, K. Keutzer et al., “Addressing the System-on-a- Chip Interconnect Woes Through Communication-Based Design”, Proceedings of 38th DAC 2001, June 2001, Las Vegas, USA.[5] F. Karim, A. Nguyen et al., “On Chip Communication Architecture for OC-768 Network Processors”, Proceedings of 38th DAC 2001, June 2001, Las Vegas, USA.[6] K. Lahiri, S.Dey et al.,”Evaluation of the Traffic Performance Characteristics of System- on-Chip Communication Architectures”, Proc. 14th Int’l Conference on VLSI Design 2001, Los Alamitos, USA.[7] L. Benini, G. De Micheli, “Network on Chip: A New SoC Paradigm”, IEEE Computer, January 2002.
  • 254 [8] T. Ye, L. Benini, G. De Micheli, “Analysis of power consumption on switch fabrics in network routers”, Proceedings of 39th DAC 2002, June 2002, New Orleans, USA. [9] S. Kumar et al., “A network on chip architecture and design methodology”, International Symposium on VLSI 2002. [10] H.-S. Wang, X. Zhu, L.-S. Peh, and S. Malik, “Orion: A Power-Performance Simulator for Interconnection Networks”, International Symposium on Microarchitecture, MICRO- 35, November 2002, Istanbul, Turkey. [11] T. Ye, G. De Micheli and L.Benini, “Packetized On-Chip Interconnect Communication Analysis for MPSoC”, Proceedings of DATE-03, March 2003, Munich, Germany, pp. 344-349. [12] J.Hu and R. Marculescu, “Exploiting the Routing Flexibility for Energy/Performance Aware Mapping of Regular NoC Architectures”, Proceedings of DATE-03, March 2003, Munich, Germany, pp. 688-693. [13] T. Grotker, S. Liao, G. Martin and S. Swan, “System Design with SystemC”, Kluwer Academic Publishers, 2002. [14] “STBus Communication System: Concepts and Definitions”, Reference Guide, STMicroelectronics, October 2002. [15] “STBus Functional Specs”, STMicroelectronics, public web support site, http://www.stmcu.com/inchtml-pages-STBus_intro.html, STMicroelectronics, April 2003. [16] Synopsys Inc., “Core Consultant Reference Manual”, “Power Compiler Reference Manual” and “VCS: Verilog Compiled Simulator Reference Manual”, v2003.06, June 2003. [17] C. Patel, S. Chai, S. Yalamanchili, and D. Schimmel, “Power-constrained design of multiprocessor interconnection networks," in Proc. Int. Conf. Computer Design, pp. 408- 416, Oct. 1997. [18] H.Zimmermann, “OSI Reference Model – The ISO model of architecture for Open System Interconnection”, IEEE Trans. on Communication, n 4, April 1980. [19] VSI Alliance Standard, “System-Level Interface Behavioral Documentation Standard Version 1”, Released March 2000. [20] Box, George E. P. and Draper Norman Richard. Empirical model-building and response surfaces, John Wiley & Sons New York, 1987
  • 255Chapter 14ENERGY-AWARE ADAPTATIONS FOR END-TO-END VIDEO STREAMING TO MOBILE HANDHELDDEVICESShivajit Mohapatra1, Nalini Venkatasubramanian1, Nikil Dutt1 ,Cristiano Pereira2 , Rajesh Gupta21 University of California, Irvine;2 University of California, San DiegoAbstract Optimizing user experience for streaming video applications on handheld devices is a significant research challenge. In this chapter, we propose an integrated end- to-end power management approach that unifies low level architectural optimiza- tions(CPU, memory, registers), OS power-saving mechanisms(dynamic voltage scaling) and adaptive middleware techniques(admission control, transcoding, net- work traffic regulation). Specifically, we identify interaction parameters between the different levels and optimize them to reduce power consumption. With knowl- edge of device configurations, dynamic device parameters and changing system conditions, the middleware layer selects an appropriate video quality and fine tunes the architecture for optimized delivery of video. Performance results in- dicate that architectural optimizations that are cognizant of user level parame- ters(e.g. transcoded video quality) can provide energy gains as high as 57.5% for the CPU and memory, when compared to the baseline case that does not employ any energy optimization. Middleware adaptations to changing network noise levels can save as much as 70% of energy consumed by the wireless net- work interface. Our approach to multiple-level and end-to-end management of power/performance has been implemented in a framework, called FORGE. We show how FORGE can substantially enhance the user experience in a mobile multimedia application.Keywords: low-power optimization, cross-layer adaptation, power-aware middleware, FORGE project14.1 MOTIVATION Limiting the energy consumption is an important design goal for mobiledevices. Designers have explored techniques for minimizing energy usage of
  • 256most components, from CPU, network, display to peripherals of a mobile sys-tem platform. On the other hand, rapid advances in processor and wirelessnetworking technology are ushering in a new class of multimedia applications(e.g. video streaming/conferencing) for mobile handheld devices. Multimediaapplications have distinctive Quality of Service(QoS) and processing require-ments which tend to make them extremely resource-hungry. Moreover, thedevice specific attributes(e.g form factor of handhelds) significantly influencethe human perception of multimedia quality. As a result delivering high qual-ity realtime multimedia content to mobile handheld devices remains a difficultchallenge. The difficulty here is due to the fact that energy efficient delivery of mediacontent with good quality attributes requires tradeoffs across various layers ofsystem implementation and functionality - from application to system softwareto networking. Since the optimal energy conditions can change dynamically,these optimizations should also allow for dynamic adaption of system func-tionality and its performance. In order to dynamically adapt to device mobility,systems need to have a high degree of “network awareness" (e.g. congestionrates, mobility patterns etc.) and need to be cognizant of a constantly changingglobal system state. Efforts are underway to exploit multimedia specific char-acteristics to enable a range of energy optimization techniques that adapt to, andoptimize for, changes in application data (video stream), OS/Hardware (CPU,Memory, Reconfigurable logic), network (congestion, noise, node mobility),residual energy (battery) and even the user environment (ambient light, sound). These issues have been aggressively pursued by researchers and numerousinteresting power optimization solutions have been proposed at various crosscomputational levels. For instance, a sampling of optimizations across designdomains are: system cache and external memory access optimizations [1, 15],dynamic voltage scaling(DVS) [29, 4], of the CPU, dynamic power manage-ment of disks and network interfaces(NICs) [8, 3], efficient compilers and ap-plication/middleware based adaptations for power management [22]. Interest-ingly, power optimization techniques developed for individual components ofa device have remained seemingly incognizant of the strategies employed forother components. While focussing their attention to a single component, re-searchers make a general assumption that no other power optimization schemesare operational for other components. However, the cumulative power gains forincorporating multiple techniques can be potentially significant. This requirescareful evaluation of the trade-offs involved and the customizations required forunified operation [21]. The interaction between different layers is even moreimportant in distributed applications where a combination of local and globalinformation helps and improves the control decisions (power, performance andQoS trade-offs) made at runtime.
  • 257 For the mobile multimedia applications, Fig. 14.1 presents the different com-putation levels in a typical handheld computer and shows the cross layer inter-actions for optimized power and performance deliverance. Video Player Other Tasks Client1 Applications Network Admission Transcoding Management Control Middleware Server Clienti DVS Scheduler Clientn Operating System Network CPU Display Cache Memory RegFiles Card H/W Figure 14.1. Abstraction Layers in Distributed Multimedia Streaming The FORGE project aims to study the tradeoffs between power, performanceand Quality of Service requirements across the various computational layers [6].The goal of FORGE is to develop and integrate hardware based architecturaloptimization techniques with high level operating system and middleware ap-proaches (Fig. 14.1), for improvements in power savings and the overall userexperience, in the context of video streaming to a low-power handheld device.Multimedia applications heavily utilize the biggest power consumers in mod-ern computers: the CPU, the network and the display(Fig. 14.1). Therefore, inFORGE, we aggregate the hardware and software techniques that lead to powersavings for these resources. To maximize power gains for a CPU architecture,we identify the predominant internal units of the architecture that contribute topower consumption. We use higher-level knowledge of the application such asquality and encoding parameters of the video stream to optimize internal cacheconfigurations, CPU registers and the external memory accesses. Similarly,we utilize hardware/design level data (e.g. cache configuration) and user-levelinformation (video quality perception) to optimize middleware and OS compo-nents for improved performance and power savings - through effective videotranscoding, power-aware admission control and efficient network transmis-sion. We reduce the power consumption of the network card by switching itto the “sleep" mode during periods of inactivity. An efficient middleware isused to control network traffic for optimal power management of the networkinterface. To maximize the user experience, we have studied video quality andpower trade-offs for handheld computers. These results drive our optimizationefforts in FORGE at each computing level.14.2 RELATED WORKLet us briefly review the optimization techniques used at various levels, suchas architecture, OS, middleware and application in the context of multimedia
  • 258applications. We then examine the relationship of FORGE with prior andongoing approaches in power aware middleware.14.2.1 Architectural AdaptationsTo provide acceptable video performance at the hardware level, efforts haveconcentrated on analyzing the behavior of the decoder software and devisingeither architectural enhancements or software improvements for the decodingalgorithm. Until recently it was believed that caches can bring no potentialbenefit in the context of MPEG (video) decoding. In fact, due to the poor lo-cality of the data stream, many MPEG implementations viewed video data as“un-cacheable" and completely disabled the internal caches during playback.However, Soderquist and Leeser showed that video data has sufficient localitythat can be exploited to reduce cache-memory traffic by 50 percent or morethrough simple architectural changes [28]. A different way of improving cacheperformance by reordering frame traversal was proposed in [9]. Register filereconfiguration was applied in [1]. [16] proposes a technique for combining twohardware adaptations (architecture adaptation and dynamic voltage scaling) toreduce energy in multimedia workloads. The algorithm presented chooses be-tween one of the two adaptations or a combination, depending on their relativeperformance. This approach is similar to ours, in that architectural optimiza-tions are combined with dynamic voltage scaling (DVS). However, instead of aframe-based adaption founded on profiling and prediction, we target tuning anarchitecture through available architectural parameters to specific video qualityrequirements. We apply the optimizations globally (for the entire period that amedia of constant quality levels are played), rather than at frame granularity.14.2.2 Operating System & Middleware AdaptationsMost power optimization efforts at the operating system level, have been fo-cussed on techniques like dynamic voltage scaling(DVS) [29, 25, 18], anddynamic power management (DPM) [17, 7]. DVS exploits the fact the CMOSlogic used in most current processors has a voltage dependent maximum operat-ing frequency. So when used at lower frequencies, the processor can operate ata correspondingly lower voltage, thereby saving battery power. The challengehere is to accurately predict workload execution times for future jobs. Whileworkloads can be predicted heuristically for best-effort applications [29], orbased on worst case execution times of real-time applications [25], worst-casebased approaches will almost certainly result in sub-optimal solutions, whereasheuristic predictions can cause timing violations for multimedia tasks. In theGRACE project, the authors suggest using an aggregate statistical demand ofapplications to adjust frequency/voltage for the processor [31]. DVS techniquesfor reducing energy in MPEG decoding has been studied in [20]. Additionally,
  • 259scheduling techniques like DSRT [5] have been studied to deliver real-timeguarantees. At the OS/middleware levels, another primary focus has been to optimizenetwork interface power consumption [8, 2, 3]. A thorough analysis of powerconsumption of wireless network interfaces has been presented in [8]. ECOSys-tem [32] is an OS level prototype that incorporates energy allocation and ac-counting mechanisms for various power consuming devices. ECOSystem usesthe Currentcy [33] model which is an abstraction for formulating energy awarepolicies. Chandra et al. [2] have explored the wireless network energy consumptionof streaming video formats like Windows Media, Real media and Apple QuickTime. Chandra and Vahdat have explored the effectiveness of energy awaretraffic shaping closer to a mobile client [3]. In [26], Shenoy suggests performingpower friendly proxy based video transformations to reduce video quality inreal-time for energy savings. They also suggest an intelligent network streamingstrategy for saving power on the network interface. FORGE uses a similarapproach, but models a noisy channel. Caching streams of multiple qualitiesfor efficient performance has been suggested in [10]. PowerScope [12] is an interesting tool that maps energy consumption to pro-gram structure. It first profiles the power consumption and system activity ofa computer and then generates an energy profile from this data. Odyssey [22]presents an applications aware adaptation scheme for mobile applications. Inthis approach the system monitors resource levels, enforces resource alloca-tion and provides feedback to the applications. The application then decideson the best possible adaptation strategy. In our approach we try to integratethe the positive aspects of all the three levels: OS, middleware and applica-tion. Application based adaptation will therefore enhance the performance ofour framework. However, applications have to be specifically designed forthe framework. JouleTrack [27] is a web-based energy measurement tool forprofiling software energy consumption of applications based on StrongArmprocessor.14.2.3 Cross-Layer Adaptation FrameworksFor efficient coordination and management of cross-layer adaptations, it is cru-cial to develop efficient resource allocation mechanisms. Q-RAM [19] modelsQoS management as a constraint optimization problem for maximizing sys-tem utility while guaranteeing minimum resources to each application. Pup-peteer [11] presents a middleware framework that uses transcoding to achieveenergy gains. Using the well defined interface of applications, the frameworkpresents a distilled version of the application to the user, in order to draw en-
  • 260ergy gains. EQoS [24] formulates energy-aware QoS adaptation as a constraintoptimization problem and solves it using heuristic algorithms. The GRACE project [31, 30] uses cross-layer adaptations for maximizingsystem utility at lower energy costs. They suggest both coarse grained and finegrained tuning through global co-ordination and local adaptation of hardware,OS and application layers. The coarse/global adaptations are expensive andless frequent and occur only when global system changes are triggered (e.gtask-set changes). The local adaptations are for the local variation in the ex-ecution of tasks. In GRACE, the global and local coordinators exist on thelocal device and perform the necessary adaptations. GRACE first tries to de-liver highest utility for each application and then optimizes the energy usingdynamic voltage scaling. In contrast, FORGE uses a proxy based distributedmiddleware approach, that integrates cross-layer(architecture, OS, middleware,application) adaptations on the local device with distributed adaptations suchas adaptive traffic shaping and transcoding at the proxy for energy gains. Whileadaptations in GRACE are limited to the local mobile device, our frameworkdesign uses a distributed middleware layer to exploit global system knowledge(e.g. device mobility patterns, network noise levels etc.) to facilitate effectivepower management (e.g. wireless NIC). Moreover, we adopt an end-to-end ap-proach to power optimization, where residual battery power of a mobile devicealso drives the adaptations. GRACE on the other hand provides a best-effortapproach to energy optimization. Additionally, FORGE tries to tune architec-tural level parameters (e.g. cache configurations) to perform optimally for thecurrently executing application. The distributed middleware co-ordinates theadaptations at each level based on a rule-base and control information from theproxy.14.3 SYSTEM MODELOur system model for a wireless mobile multimedia distributed system is shownin Fig. 14.2. The system entities include a multimedia server, a proxy serverthat utilizes a directory service, a rule base for specific devices and a videotranscoder, an ethernet switch, the wireless access point and users with low-power wireless devices. The multimedia servers store the multimedia contentand stream videos to clients upon receipt of a request. The users issue requestsfor video streams on their handheld devices. All communication between thehandheld device and the servers are routed through the proxy server, that cantranscode the video stream in realtime. The middleware executes on both thehandheld device and the proxy, and performs two important functions. On thedevice, it obtains residual energy availability information from the underlyingarchitecture and feeds it back to the proxy and relates the video stream parame-ters and network related control information to lower abstraction layers. On the
  • 261proxy, it performs feedback based power aware admission control and realtimetranscoding of the video stream, based on the feedback from the device. It alsoregulates the video transmission over the network based on the noise level andthe video stream quality. Additionally, the middleware exploits dynamic globalstate information(e.g mobility info, noise level etc.) available at the directoryservice and static device specific knowledge (architecture, OS, video qualitylevels) from the static rule base, to optimally perform its functions. The rateat which feedback are sent by the device is dictated by administrative policies(e.g. periodic feedback). Moreover, we assume that network connectivity ismaintained at all times. Rule Directory base Service noise Transcoder C USERS C S P Server Proxy Switch Access C Point WAN WIRED ETHERNET W I R E L E SS Figure 14.2. System Model In the rest of the chapter, we present important research challenges encoun-tered at each level and discuss approaches that involve both distributed proxybased adaptations coupled with coordinated cross-layer energy optimizationsat the device.14.4 HARDWARE/ARCHITECTURAL LEVEL OPTIMIZATIONS The architectural optimizations are particularly important because of the useof microelectronic system-on-chip components used in multimedia platforms.Since most multimedia applications spend a significant amount of time access-ing and transforming audio and video data, the design of the memory subsystemarchitecture, and compiler support for exploiting the specialized memory struc-tures are critical for meeting the performance, power and cost budgets of suchapplications. Since the memory subsystem will dominate the cost (area), performance andpower, we have to pay special attention to how it can benefit from customiza-tion. For example, the memory can be selectively cached; the cache line sizecan be determined by the application; the designer can opt to discard the cachecompletely and choose specialized memory configurations such as FIFOs andstream buffers. The exploration space of different possible memory architec-tures is vast, and there have been attempts to automate or semi-automate thisexploration process [13].
  • 262 Data Register Display Memory Cache File Network CPU card Functional Clock Units a b Figure 14.3. Main Components of a Handheld Device (a) and CPU Detail (b)14.4.1 Hardware-level Optimizations for Handheld Devices There are three major sources of power consumption in a handheld devicesuch as a Compaq iPAQ 3650 for which we indicate the corresponding powernumbers: display (approximately 1W for full backlight), network hardware(1.4W) and CPU/memory (1-3W, with the additional board circuits). Each ofthese subsystems also provide opportunities for controlling the power dissi-pation. In case of the display (LCD), the main energy drain comes from thebacklight, which is a predefined user setting and therefore has a limited degreeof controllability by the system (without affecting the final utility). The networkinterface allows for efficient power savings if cognizant of the higher level pro-tocol’s behavior and will be explored in a subsequent section. Out of the threecomponents mentioned above, the CPU coupled with the memory subsystemposes the biggest challenge. The dependence on the input data to be processed,the quality of the code generated by the compiler and the organization of itsinternal architecture make predicting its power consumption profile very hardin general; nevertheless, very good power saving results can be obtained byutilizing the knowledge of the application running on it and through extensiveprofiling of a representative data input set from the application’s domain. Inthe rest of this section, we focus our attention on the possible optimizations atthe CPU level for a multimedia streaming application (e.g MPEG-1). We identified the subcomponents of the CPU (Fig. 14.3(b)) that consumethe most power and observed the power distribution inside the CPU for MPEGdecoding. By running the decoder process in a power simulator (Wattch) forvideos of various types and by measuring the relative power consumption ofeach unit in the CPU we generate the internal processor power distribution. Weconclude that: • The relative power contribution of the internal units of the CPU do not varysignificantly with the nature or quality of the video played. A possible reasonfor this is the symmetrical and repetitive nature of MPEG decoding, whoseprocessing is done on fixed size blocks or macroblocks.
  • 263 • The units that show an important contribution to the overall power con-sumption and are amenable for power optimization are: caches, register filesand functional units. Cache behavior greatly affects the memory performanceand hence power consumption, so we optimize the entire memory subsystemin an integrated way. We briefly discuss these components, their impact on overall power con-sumption and how it can be affected by these architectural choices:• Caches/Memory: cache configurations are determined by their size, numberof sets, and associativity. The size specifies how large a cache should be, whilethe associativity/number of sets control its internal structure. We identify thatmost power gains for MPEG are possible through reconfiguration of the datacache and its effect on the memory traffic, thus amplifying the effect of poweroptimizations through cache reconfiguration.• Frame Traversal: Decompressing MPEG video in its implied order doesnot leave space for exploiting the limited locality existent between dependentmacroblocks. By just changing the frame traversal order algorithm based onthe existing locality, faster decompression rates and higher power savings areachieved via reduced memory accesses [9]. Our proxy-based approach allowsfor a transparent on-the fly traversal reordering at the proxy server. In additiondynamic voltage scaling provides further savings for MPEG streaming as itallows tradeoffs for transforming the frame decoding slack time (CPU idle time)into important power savings. We discuss DVS and investigate the implicationsof DVS on other power optimizations in the system. All these parameters whenfine-tuned for a specific video quality, will provide the best operating point(forpower and performance) for a specific video stream.14.4.2 Quality-driven Cache Reconfiguration Power consumption for the cache depends on the runtime access counts:while hits result in only a cache access, misses add the penalty of accessingthe main memory (external). Fortunately, in most applications the inherent lo-cality of data means that cache miss rate is relatively low and so are accessesto external memory. However, MPEG decoding exhibits a relatively poor datalocality, which, when combined with the large data sets exercised by the algo-rithm, leads to an increase in the cache memory-traffic. In order to find the bestsolution point, we resort to extensive simulation and profiling with data that isrepresentative of the video domain. Internal CPU caches are characterized bytheir size(S), number of sets(N S), line size(LS) and associativity(A). Ourcache reconfiguration goal is optimizing energy consumption for a particularvideo quality level Qk. In general, cache power consumption for a particularconfiguration and video quality is given by the function Ecache,k(S, A). Byprofiling this function for the entire search space (S, A) of available cache
  • 264 Total Energy (J) 1.7 1.6 1.5 1.4 1.3 4 8 1.2 16 32 1.1 1 2 4 64 Cache 8 16 32 Size Cache Associativity Figure 14.4. Cache Energy Variation on Size and Associativityconfigurations, we generate a cache energy variation graph shown in Fig. 14.4.Depending on the video quality Qk played, there will be one optimal operatingpoint for that video quality: (Sk , Aopt). We found out that for all video opt kqualities an optimized operating point exists and it improves cache power con-sumption by up to 10-20% (as opposed to a suboptimized configuration). Thistechnique effectively fine-tunes the organization of the cache so that it perfectlymatches the application and the data sets to be processed, yielding importantpower savings.14.5 OS/MIDDLEWARE LEVEL OPTIMIZATIONSGains in power reduction and performance improvement from architecturaloptimizations can be further amplified if the low-level architecture is cognizantof the exact characteristics of the streamed video. An adaptive middlewaresoftware at a proxy can dynamically intercept and doctor a video stream toexactly match the video characteristics for which the target architecture hasbeen optimized. It can also regulate the network traffic to induce maximalpower savings in a network interface. Additionally, with knowledge of thevideo stream the operating system can employ an optimized dynamic voltagescaling of the CPU.14.5.1 Integrated Dynamic Voltage Scaling For a given supply voltage, V, and clock frequency f, the dynamic powerdue to digital CMOS varies linearly with frequency and quadratically with thesupply voltage (which is also the switching voltage). This relationship can beused at the application level [4]. In our case, for MPEG decoding, frames areprocessed in a fraction of the frame delay (Fd = 1/f rame rate). The actualframe decoding time D depends on the type of MPEG frame being processed(I, P, B) and is also influenced by the cache configuration (S, A) and DVSsetting (f, V ). We assume a buffer based decoding, where the decoded frames
  • 265are placed in a temporary buffer and are only read when the frame is displayed.This allows us to decouple the decoding of the frame from the displaying part;decoding time is still different for different frames, we therefore assume anaverage D for a particular video stream/quality. The difference between theaverage frame delay and actual frame decoding time gives us the slack timeθ = Fd − D. We can then perform DVS, where we slow down the CPU touse up the slack time. Cache configuration also slightly influences the framedecoding time (due to the cache misses, which translate into external memorytraffic), extreme values proving very inefficient. An optimized cache combinedwith DVS yields the best power saving results. Determination of the bestoperating point for the DVS/cache reconfiguration requires simulation of theapplication with the power aware system software that has direct influence onthe technology parameters. This is discussed next.14.5.2 Power Aware Operating System Architecture We view the notion of power ’awareness’ in the application and OS as acapability to carry out a continuous dialogue between the application, the OS,and the underlying hardware. This dialogue establishes the functionality andperformance expectations (or even contracts, as in real-time sense) within theavailable energy constraints. We describe here our implementation of a specificservice, namely the task scheduler, that makes the OS power aware. The sched-uler architecture is composed of two software layers and the OS kernel. Onelayer interfaces applications with operating system and the other layer makespower related hardware “knobs” available to the operating system. Both layersare connected by means of corresponding power aware operating system ser-vices as shown in Figure 14.5. At the topmost level, embedded applicationscall the API level interface functions to make use of a range of services thatultimately makes the application energy efficient in the context of its specificfunctionality. The API level is separated into two sub-layers. The PA-API layerprovides all the functions available to the applications, while the other layer pro-vides access to operating system services and power aware modified operatingsystem services (PA OS Services). Active entities that are not implementedwithin the OS kernel are also be implemented at this level (threads created withthe sole purpose of assisting the power management of an operating systemservice). We call this layer the power aware operating system layer (PA-OSL). Tointerface the modified operating system level and the underlying hardware level,we define a power aware hardware abstraction layer (PA-HAL). The PA-HALprovides access to the power related hardware parameters in a way that makesit independent of the hardware.
  • 266 Application Level Applications API Level PA-API PA-OSL OS Level Scheduler POSIX Device PA OS Drivers OS Services Kernel Memory Manager Hardware Level OS HAL PA-HAL HARDWARE Figure 14.5. Power Aware Operating System Architecture14.5.3 Middleware based Network Traffic RegulationWe now describe a proxy-based traffic regulation mechanism to reduce energyconsumption by the device network interface. Our mechanism (a) dynami-cally adapts to changing network(e.g noise) and device conditions(e.g. residualbattery energy). (b) accounts for attributes of the wireless access points (e.g.buffering capabilities) and the underlying network protocol (e.g. packet size).(c) uses the proxy to buffer and transmit optimized bursts of video along withcontrol information to the device. However, even though packets are transmit-ted in bursts by the proxy, the device receives packets that are skewed over timeFig. 14.6; this cuts power savings, as the net sleep time of the interface is re-duced. The skew is caused due to the ethernet access protocol(e.g CSMA/CD)and/or the fair queueing algorithms implemented at the wireless access points.Our mechanism optimizes the stream, such that optimal video bursts sizes aresent for a given noise level, thus maximizing energy savings without perfor-mance costs. Wireless network interface(WNIC) cards typically operate in four modes:transmit, receive, sleep and idle. We estimated the power consumption ofthe Cisco Aironet 350 series WLAN card to have the following power con-sumption characteristics: transmit(1.68W), receive(1.435W), idle (1.34W) andsleep(0.184W) which agree with the measurements made by Havinga et al.in [14]. This observation suggests that considerable energy savings can beachieved by transitioning the network interface from idle to sleep mode during
  • 267periods of inactivity. The use of bursty traffic was first suggested by Chan-dra [2, 3] and control information was used for adaptation in [26]. We analyze the above power saving approach using a realistic network frame-work(Fig. 14.6), in the presence of noise and AP limitations [21]. The proxymiddleware buffers the transcoded video and transmits I seconds of video in asingle burst along with the time τ =I for the next transmission as control infor-mation. The device then uses this control information to switch the interface tothe active/idle mode at time τ + γ × DEtoE , where γ is an estimate betweenzero and one and DEtoE is the end-to-end network delay with no noise. User 1 User N C Wired C Wired wireless Proxy Wireless device P HTTP/TCP/IP 802.11b C RTP/UDP/IP Access packets Point t t Figure 14.6. Wireless Network We acknowledge that a QoS aware preferential service algorithm at the accesspoint can impact power management significantly. The above analysis can beused by an adaptive middleware to calculate an optimal I(burst length) for anygiven video stream and noise level. Note that energy overhead for buffering thevideo packets is not affected by using our strategy because the number of readand write memory operations remain unchanged irrespective of the memorybuffer size. In the previous section, we demonstrated how low level architecture canbe optimized using high level information. In this section, we presented twomiddleware techniques that can be used to compliment the low-level hardwareoptimizations, lower energy consumption of the NIC and improve the overallutility of the system. We now introduce a middleware based adaptation schemefor backlight power savings in handheld devices.14.5.4 Reducing Backlight Power Consumption The backlight accounts for considerable energy overheads in a low-powerdevice. However, potentially large energy savings are realizable by operatingthe device at a lower backlight intensity levels. We explore a more aggressiveapproach to brightness compensation and device backlight control for stream-ing video. Furthermore, the adaptation is shifted away from the low-powerdevice and performed at a network proxy server, obviating the need for thedecoder on the device to be modified. We have found that aggressive bright-ness compensation is possible for streaming video as compared to still images,
  • 268without considerably impacting the video quality. This is because small defects(introduced due to aggressive compensation) that might be noticeable in a stillimage are less discernable in streaming video where several frames (images) aredisplayed on the screen every second. We also propose an effective brightnesscompensation algorithm for optimized power savings [23]. In this approach,we introduce middleware based adaptation schemes which integrate our com-pensation algorithm to achieve low power backlight operation for streamingvideo content to mobile handheld devices. Our experiments indicate that thisapproach can provide power reductions of up to 60% of the power consumptionattributed to the backlight, depending on the chosen adaptation scheme and thecharacteristics of the streamed video. We assume that the proxy server has access to a database of profiled lumi-nosity values for various video streams and device specific parameters (e.g.number of backlight levels, average luminosity at each level etc.), a rule baseto determine compensation values and a video transcoder(Fig. 14.7); and low-power wireless devices capable of displaying streaming MPEG video content.All communication between the handhelds and the multimedia server are routedthrough the proxy server that can change the video stream in real-time. Figure 14.7. Model for Backlight Adaptation Each device/client has an application layer where the video stream is de-coded and a middleware layer which routes the information flowing to andfrom the video decoder application. The client middleware layer has accessto system parameters such as the backlight levels, the current battery level andinformation identifying the type and make of the handheld (e.g. iPAQ, Jor-nada etc). In addition to accessing these system parameters, the middlewarelayer on the client can change these parameters (e.g. operating backlight level)through API calls to the underlying OS. The middleware on the proxy performsthe dynamic adaptation of the streaming video content (brightness compensa-tion) and communicates control information to the client middleware (operatingbacklight levels) through the low bandwidth control stream. The proxy main-tains a database of information about the videos available at the server and
  • 269information specific to different handheld types such as the number, luminousintensity and average power consumption of the backlight levels. Additionally,the proxy also employs a static rule base which specifies conditions which de-termine values for backlight and video compensation. The database and certainparameters of the rule base are populated by extensive profiling and subjectiveassessment of videos on different handhelds.14.6 APPLICATION LAYER ADAPTATIONImproving the service lifetimes of low-power mobile devices through effectivepower management strategies can facilitate optimization of user experience forstreaming video on to handheld devices. To achieve this, a system should beable to dynamically adapt to global system changes, so that the entire duration ofa requested video is streamed to the user at the highest possible quality, whilemeeting the power constraints of the user’s low-power device. We achievesuch an optimal balance between power and performance, by introducing anotion of “Utility Factor UF " for a system, and optimizing the UF for thesystem. This approach precludes the system from aggressively optimizing forpower at the expense of performance and vice-versa; thereby providing anoptimized operating point for the system at all times. UF is a measure of “usersatisfaction" and we specify it as follows: given the residual energy Eres ona handheld device, a threshold video quality level (QA : QMAX ≥ QA ≥QMIN ) acceptable to the user, and the time of the video playback T, the UFof the system is non-negative, if the system can stream the highest possiblequality of video to the user such that the time, quality and the power constraintsare satisfied; otherwise UF is negative. Let PV ID denote the average powerconsumption rate of the video playback at the handheld and QP LAY be thequality of video streamed to the user by the system. Using the above notation,we define UF as follows:   QP LAY − QMIN IFF PV ID ∗ T < ERES UF = QP LAY ≥ QA  −1 Otherwise Our experiments to determine video transcoding levels that affect the videoquality against increased energy consumption indicate the following major con-clusions:• It is hard to programmatically identify video quality parameters( a combina-tion of bit rate, frame rate and video resolution) that produced a user perceptiblechange in video quality and/or a noticeable shift in power consumption in hand-helds. • For all the video streams on handheld devices, it was enough to use justthree standard intermediate formats(e.g SIF(320x240), Half SIF(340x160) andQuarter SIF(160x120)) for frame resolution values. Other resolutions did not
  • 270Table 14.1. Energy-Aware Transformations for Compaq Ipaq 3650 with bright backlight, Cisco350 Series Aironet WNIC card. (Q1) Terrible, (Q2) Bad, (Q3) Poor, (Q4) Fair, (Q5) Good, (Q6)Very Good, (Q7) Excellent, (Q8) Like Original Quality Parameters Avg. Power Avg. Power (WinCE) (Linux) (Q8) SIF, 30fps, 650Kbps 4.42W 6.07W (Q7) SIF, 25fps,450Kbps 4.37W 5.99W (Q6) SIF, 25fps, 350Kbs 4.31W 5.86W (Q5) HSIF, 24fps, 350Kbps 4.24W 5.81W (Q4) HSIF, 24fps, 200Kbps 4.15W 5.73W (Q3) HSIF, 24fps, 150Kbps 4.06W 5.63W (Q2) QSIF, 20fps, 150Kbps 3.95W 5.5W (Q1) QSIF, 20fps, 100kbps 3.88W 5.38Wproduce a perceptible quality change or power uptake compared to the nearestSIF encoded video with similar bit and frame rates. Based on these conclusions, we identified eight dynamic video stream trans-formation parameters (Table 14.1) for our proxy-based realtime transcoding anduse the profiled average power consumption values to perform our adaptations.14.7 SUMMARYIt has been pointed out by several researchers that power optimization acrossvarious levels of system functionality and implementation (architecture, OS,middleware, application) can lead to much greater savings than the case whenthese are individually optimized for power. The challenge is how these op-timizations can be coordinated across layers; what is the right architecturalframework that allows this optimization to occur simultaneously and even dy-namically? To answer this question, this paper proposes a proxy-based middle-ware solution to accommodate optimizations across diverse clients with limitedcomputation and battery power by controlling the amount of needed computa-tion and communication to the client device. We showed how such adaptation inthe middleware can be used to improve energy efficient delivery of multimediacontent in the case of streaming video. User perception of video also plays avital role in deciding the proxy-based video transformations and in identifyingarchitectural tuning “knobs". However, identifying the various video qualitiesremains a highly subjective aspect of the study. Identifying video quality levelsobjectively/programmatically still remains an open research challenge. In prac-tice however, the widespread deployment of such a unified power managementframework for mobile devices would require a set of APIs (programming inter-faces) to be implemented at the various computational layers; this API should fa-
  • 271cilitate effective communication between the various levels. Recent approachestowards power management suggest a more open and flexible architecture formobile devices that allows higher layers to make informed adaptations at lowerlayers and vice-versa. A prototype implementation of the framework is cur-rently underway as a part of the FORGE(http://www.ics.uci.edu/˜ orge) project. fAcknowledgments This work was supported by funding from an ONR MURI Grant N00014-02-1-0715 and NSF NGS award ACI-0204028.References[1] Azevedo, Ana, Cornea, Radu, Issenin, Ilya, Gupta, Rajesh, Dutt, Nikil, Nicolau, Alex, and Veidenbaum, Alex (2001). Architectural and compiler strategies for dynamic power management in the copper project. In Inter- national Workshop on Innovative Architecure.[2] Chandra, Surendar (2002). Wireless Network Interface Energy Consump- tion Implications of Popular Streaming Formats. In Multimedia Computing and Networking.[3] Chandra, Surendar and Vahdat, A. (2002). Application-specific Network Management for Energy-aware Streaming of Popular Multimedia Formats. In Usenix Annual Technical Conference.[4] Choi, Kihwan, Dantu, Karthik, Chen, Wei-Chung, and Pedram, Massoud (2002). Frame-Based Dynamic Voltage and Frequency Scaling for a MPEG Decoder. In International Conference on Computer Aided Design.[5] Chu, H. H. and Nahrstedt, Klara (1999). "Cpu Service classes for multime- dia applications". In International Conference on Multimedia Computing and Systems.[6] Cornea, Radu, Dutt, Nikil, Gupta, Rajesh, Mohapatra, Shivajit, and et. al. (2003). ServiceFORGE: A Software Architecture for Power and Quality Aware Services. In FME Symposium.[7] Douglis, Fred, Krishnan, P., and Marsh, B. (1994). Thwarting the power hungry disk. In WINTER USENIX conference.[8] Feeney, L.M. and Nilsson, M (2001). Investigating the Energy Consumption of a Wireless Network Interface in an Ad Hoc Networking Environment. In IEEE Infocom.
  • 272[9] Feng, Wu-chi and Sechrest, S. (1996). Improving data caching for software mpeg video decompression. In IS&T/SPIE Digital Video Compresssion: Algorithms and Technologies.[10] Flinn, J. and Satyanarayanan, M. (1999a). Energy-Aware Adaptations for Mobile Applications. In In Symposium on Operating Systems Principles.[11] Flinn, Jason, de Lara, Eyal, Satyanarayanan, M., Wallach, Dan S., and Zwaenepoel, Willy (2001). “Reducing the energy usage of office appli- cations". In IFIP/ACM International Conference on Distributed Systems Platforms.[12] Flinn, Jason and Satyanarayanan, M. (1999b). PowerScope: a tool for pro- filing the energy usage of mobile applications. In Second IEEE Workshop on Mobile Computing Systems and Applications.[13] Grun, P., Dutt, N., and Nicolau, A. (2003). "Memory architecture ex- ploration for programmable embedded systems". In Kluwer Academic Publishers, Norwell, MA.[14] Havinga, Paul J. M. (2000). Mobile Multimedia Systems. PhD thesis, University of Twente.[15] Hughes, C. J., Srinivasan, J., and Adve, S. V. (2001a). Saving energy with architectural and frequency adaptations for multimedia applications. In IEEE/ACM International Symposium on Microarchitecture.[16] Hughes, C. J., Srinivasan, J., and Adve, S. V. (2001b). Saving energy with architectural and frequency adaptations for multimedia applications. In IEEE/ACM International Symposium on Microarchitecture.[17] Irani, S., Shukla, S., and Gupta, R. (2002). “Competitive analysis of dynamic power management strategies for systems with multiple power saving states". In Design Automation and Test in Europe.[18] Kumar, Pavan and Srivastava, Mani (2000). Predictive Strategies for Low- Power RTOS Scheduling. In International Conference on Computer De- sign.[19] Lee, C., Lehoczky, J., Siewiorek, D., Rajkumar, R., and et.al (1999). A Scalable solution to the multi-resource QoS problem. In Real-Time Systems Symposium.[20] Mesarina, M. and Turner, Y. (2002). A Reduced Energy Decoding of MPEG Streams. In Multimedia Computing and Networking.[21] Mohapatra, Shivajit, Cornea, Radu, Dutt, Nikil, Nicolau, Alex, and Venkatasubramanian, Nalini (2003). Integrated power management for video streaming to mobile handheld devices. In ACMMM.[22] Noble, B. D., Satyanarayanan, M., D.Narayanan, J.E.Tilton, and Flinn, J. (1997). Agile Application-Aware Adaptation for Mobility. In In Symposium on Operating Systems Principles.
  • 273[23] Pasricha, Sudeep, Mohapatra, Shivajit, and et. al. (2003). "Reducing back- light power consumption for streaming video applications on mobile hand- held devices". In ACM/IEEE/IFIP Workshop on Embedded Systems for Real-Time Multimedia, 2003.[24] Pillai, P., Huang, H., and Shin, K. G. (2003). Energy-Aware quality of Service adaptation. In Technical Report CSE-TR-479-03, Univ. of Michi- gan.[25] Pillai, P. and Shin, K. G. (2001). Real-Time Dynamic Voltage Scaling for Low-Power Embedded Operating Systems. In In Symposium on Operating Systems Principles.[26] Shenoy, Prashant and Radkov, Peter (2003). Proxy-Assisted Power- Friendly Streaming to Mobile Devices. In Multimedia Computing and Networking.[27] Sinha, Amit and Chandrakasan, Anantha (2001). "jouletrack - a web based tool for software energy profiling". In Design Automation Conference.[28] Soderquist, Peter and Leeser, Miriam (1997). Optimizing the data cache performance of a software MPEG-2 video decoder. In ACM Multimedia, pages 291–301.[29] Weiser, M., Welch, B., Demers, A., and Shenker, S. (1994). Scheduling for Reduced CPU Energy. In In Symposium on Operating Systems Design and Implementation.[30] Yuan, W. and Nahrstedt, K. (2004). Process Group Management in Cross- Layer Adaptation. In Multimedia Computing and Networking.[31] Yuan, W., Nahrstedt, K., Adve, S., Jones, Doug., and Kravets, Robin (2003). Design and Evaluation of a Cross-Layer Adaptation Framework for Mobile Multimedia Systems. In Multimedia Computing and Networking.[32] Zeng, H., Ellis, C., Lebeck, A., and Vahdat, A. (2002). "Ecosystem: Man- aging energy as a first class operating system resource". In Architectural Support for Programming Languages and Operating Systems.[33] Zeng, H., Ellis, C., Lebeck, A., and Vahdat, A. (2003). Currentcy: Unifying policies for resource management. In USENIX.