Download to read offline
Uploaded byHideaki Hata
Using High-Rising Cities to Visualize Performance in Real-Time
The document discusses a technical paper that proposes using high-rising city visualizations to monitor program performance in real-time. It addresses challenges in profiling applications, specifically the limitations of existing tools and the difficulties in reproducing scenarios, and showcases a solution that provides constant profiling and an innovative visual format. The tool aims to enhance user experience by identifying bottlenecks and uncovering unreproducible performance bugs.
Using High-Rising Cities to Visualize Performance in Real-Time
- 1. Using High-Rising Citiesto Visualize Performance in Real-Time Katsuya Ogami Raula Gaikovina Kula Hideaki Hata Takashi Ishio Kenichi Matsumoto VISSOFT 2017 Technical Paper
- 2. Why Measure ProgramPerformance? Improve User Experience Understand Bottlenecks 2
- 3. Existing Difficulties 1. Limitation ofVisible Information... Major real-time Java profiler VisualVM 3
- 4. Existing Difficulties 2. Troublesome Reproducing a SpecificScenario Execute Find scenario(s) Reproduce and Profile 4
- 5. Our Solution: ・CodeCity styleof visualization ・Constant profiling (no scenario) 5 (Like audio player visualization)
- 6. 6 Layout of ourvisualization Performance with building elevationCode city for structural properties
- 7. 7 Building height: ratioof execution time in L (1 sec)
- 8.
- 9. 9 Tetris Application VisualizationScreen Demo: https://www.youtube.com/watch?v=eleVo19Hp4k
- 10. Case 2: 2D ActionGame (MarioLike) Encountered a short freeze bug 10 https://github.com/aidiary/javagame
- 11.
- 12. Take Home Message: OurTool is useful for: ● Constant monitoring of executed methods. ● Discover unreproducible performance bugs. 12
- 13. Current Work inProgress We are now investigating how to profile Android applications 13
- 14.
Editor's Notes
- #2 Good Afternoon, My name is Katsuya Ogami, I am a master student from Nara Institute of Science and Technology in Japan. Today, I am happy to present our paper “Using High-Rising Cities to Visualize Performance in Real-Time”. This work was also co-authored by Raula Kula, Hideaki Hata, Takashi Ishio and Kenichi Matsumoto. Lets start!
- #3 So why measure Program Performance? As our system evolves over time, so does its complexity. In this case, developers need to monitor and understand its performance for improving user experience for better software. A developer can use profiling tools to measure performance. A profiler allows a developer to understand important information related to performance, especially for issues such as bottlenecks during execution time. So to make our software faster and more user friendly.
- #4 Although existing tools are very advanced, we find that conventional profiling tools suffer from two limitations: First, we find that there is a limit to visible information. Let me show you these problems in detail. (click video) In this video, we can see that the CPU profiling of major real-time Java profiler VisualVM. We see this typical visualization of execution time is the form of a bar chart. We see that other important information are not clearly visible in this single interface: (point to different code) we cannot see the relationship between the different components of the profiled program. Furthermore, you can see that the profiler is quickly accumulating all data from start to end, making it difficult for constant monitoring of this profiled program.
- #5 The second and more severe limitation is the effort needed to reproduce specific scenarios for profiling a program. What this means is that, as a developer, you must tell the profiler when to start and stop data collection. Developers may need to reproduce an exact scenario, sometimes this can include numerous tiny iterations to an existing scenario. Sometimes it is almost impossible to reproduce a bug, as you need to find the exact scenario. (wait) Therefore, we see that making a scenario can be a super time-consuming and tedious job for the developer. So what is our solution!
- #6 We now present our solution to attempt to solve the two limitations we stated. Our prototype is a Java Profiling tool that uses an alternative method of profiling. To solve the first issue of visibility: We adopt the CodeCity visualization to show more intuitive relationships between executed portions of the code. Unlike conventional profiling user interfaces, CodeCity can quickly show which code is being executed concurrently across large portions of code. (show the bumps in the cities). This executed code is shown like an audio player visualization, rising and falling as its been played. Since, buildings can quickly change heights, we call it rising cities. To solve the second issue of scenarios: Our profiler does not require a scenario. That means no start and stop data collection, since the execution is in real-time. In this case, you can dynamically make a new scenario as you profile the program.
- #7 We now discuss the typical layout of our visualization. The visualization is divided into two levels. Firstly, we intuitively express the structural properties of the profiled program, We use a district localization on the horizontal plane as a very simple view. (point to block) You can see that every method is represented by uniform square-sized block. To show relationship of the program, we use the typical structure of blocks, with methods grouped according to the class, and classes are grouped according to the package. In the figure, we use color to differentiate, that is, orange classes are placed on a green package, with the methods placed on a class blocks. Secondly, to express the program performance, we use the blue building elevation on the vertical axis. As shown In the figure, the execution time is expressed by the height of the method block. Next, we will explain how we calculate the elevation of this rising building.
- #8 The main goal of the building elevation, is to show at almost real-time, which methods are being executed. Our key idea is to spread the execution time across all executed methods during a fixed time frame, in this case L. Simply, we split the time to execute each method. So for our example (point to the different points in the figure), at this time L, method A was executed from t1 to t4, while method C was executed from t2 to t3. In our prototype, we find that setting the L value to 1 second is sufficient to give a close to real-time experience of the methods been executed.
- #9 To evaluate the usefulness of our prototype, we profiled two real-world applications. In both cases, we use gaming applications, because games can have a greater user experience and visibility, and games have numerous scenarios in their executions. I now will introduce the first case. We chose the Tetris application due to its simple functionality and ease of use. So Lets profile the rising cities of Tetris.
- #10 Now we show how our profiler uses the CodeCity design to show a program structural properties. (point to left package) The left package is an external library of Tetris. (point to right package) And the right is the local package that implements the game. Now, since the program is not being executed, there is no buildings in the city. Lets start Tetris... (start Tetris) When we start the game, as each method is been executed, we observe the blue buildings starting to rise and fall in the city. (zoom and mouse on “run()”, show info panel) With mouse over, we can see more detailed information. (end tetris) (mouse on “run()”) However, we can see that the run method building still remains with an elevation at 100%. This means that the thread is looping in run method, and is not dying. (restart tetris) When we restart the game, (mouse on “run()”, show thread num) we find that the threads have increased. So we have two threads. We find this pattern, that after each restart, a new thread is always being created. We conclude that this application has thread leak, which if untreated, will finaly result in an OutOfMemoryError bug.
- #11 Next in case 2, we will profile another gaming application. This is a 2D action game, called MarioLike. In this scenario, we show a freezing performance bug.
- #12 In this video, we show a scenario where the program experiences two very quick program freezes. We will play this video two times, so at first please attention to the left game screen. (point left) (play video) The first is when get a coin. The second is when knock a box. (wait for freezes) (call) First. Second. Lets see again. Next, please pay attention the right visualization screen. (point right) (roll back) (call, point two methods) First. Second. Now, you can see the two methods which caused each freeze. We find that these types of program bugs have low reproducibility, which makes them difficult to detect and monitor, for existing profilers. Since our profiler is running in real-time, a developer can easily find the method responsible for causing such performance bugs.
- #13 Now our take home message. As shown in the two cases, our profiler is constantly monitoring various methods in a program. Also, the CodeCity visual design allows the user to intuitively understand each concurrent method to quickly discover any performance issues. During our cases, we found that our profiler discovers issues and bugs that is difficult to reproduce.
- #14 In our latest work, we have moved to profiling Android applications. Furthermore, our profiler should scale to larger and more complex programs.
- #15 To end the talk, I would like to present our artifact for this paper. You can get our profiler from this GitHub page. It is super easy to use, as long as you have the Java Runtime Environment. Again thank you for your time and this is the end of my presentation.