Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Visualization of Complex Systems


Published on

This is a retrospective of my 30 year career in Software Engineering. Most of the tools I worked on have to do with Java, Eclipse, Python, visualization, profiling, and people.

Published in: Software
  • Login to see the comments

Visualization of Complex Systems

  1. 1. Visualization of Complex Systems Chris Laffra 1989 - 2019
  2. 2. Who is Chris Laffra? Chris works at Uber Amsterdam, since Jan 2018. Chris has worked in Tech at IBM Research, Morgan Stanley, IBM Rational, Bank of America, and Google. At Uber, Chris works on tools that increase Uber’s ability to roll out software faster and more frequently. Chris has spent the last 30 years of his life on a mission to make other engineers more productive. See for more details.
  3. 3. Goals This is an overview of my career. Over the last 30 years, I have spent most of my technical career towards understanding, profiling, and visualizing complex systems.
  4. 4. System “Related entities forming a unified whole” First, what is a system?
  5. 5. Complex “No centrally defined interactions” “Consisting of many parts” “Complicated” Second, what is complexity?
  6. 6. Cynefin framework
  7. 7. “Images, diagrams, or animations communicating a message.” “A picture is worth a thousand words” Visualization Finally, what is visualization?
  8. 8. PROCOL How I got started, by visualizing the execution of applications written in PROCOL, my PhD thesis language. Book #1 Book #2
  9. 9. Explore the heap of objects, starting with the “main” object. Understand the shape of an object See live calls to object methods And argument values 1989 The goal: Understand what is happening and see surprises.
  10. 10. HotWire My first job after my PhD. Developed a system for custom visualizations for C++ and Smalltalk.
  11. 11. the subject Showing live objects, methods, & patterns
  12. 12. Original source code Custom, declarative visualization script. The result 1994
  13. 13. JVM As soon as Java came out, I wanted to learn more about it. So, I ended up writing book the fourth book that was published on Java while working at Morgan Stanley. Book #3
  14. 14. At IBM Research, I worked on Jax, which used JikesBT, that I developed, to analyze whole program call graphs and optimize Java programs for size. For example, we reduced javac to half its size. Aside from being smaller, the secondary benefit was that applets loaded faster. 1999
  15. 15. 2001 I wrote the first AdBlock, 8 years before the actual AdBlock was launched. I instrumented the entire IE Java runtime, using JikesBT to do bytecode instrumentation, and added specific hooks for networking, general profiling, and ability to stop/start/kill any applet remotely.
  16. 16. Eclipse Book #4 My IBM/OTI team in Amsterdam developed the first commercial IBM product based on Eclipse, focusing on embedded Java. We constantly worried about size and performance. The learnings from Jax were applied to make embedded Java apps as small as possible. I also started looking into performance of larger Java programs, such as Eclipse itself. To better understand how Eclipse works, I signed up to co-author a book on Eclipse and wrote numerous profilers.
  17. 17. One experiment was a heap-exploration tool, in this case showing all the objects in Eclipse pointing to a given editor. 2004
  18. 18. A custom CPU and memory analysis tool I developed for Eclipse was XRay. The goal was to show patterns and for engineers to discover surprising patterns related to performance. 2004
  19. 19. Other tools, such as Eclipse Universe, offered novel structural communication patterns between Eclipse plugins, allowing engineers to dive deeper into specific scenarios. It showed domain-specific issues, in addition to resource management issues and challenges. 2004
  20. 20. Finally, working with IBM Research, I devised novel way to package up Eclipse applications using streaming delivery, combined with introducing memory-mapped shared classes using the J9 JXE format. This allowed us to save about ⅓ in startup time and memory used. 2005 WebSphere VMs per Node Normal JXEs My impact. 2X scaling.
  21. 21. Python Book #5 At Bank of America, I was the first hire on the Quartz project. My interest on this project was to increase other people’s understanding in how the platform was structured and implemented. I developed a live-coding training system and contributed to coding workflows, training, and general developer advocacy. Quartz Academy Chris Laffra Bank of America
  22. 22. 2012 At BofA, I developed a live coding system similar to Jupyter notebooks. A novel invention I added was the ability to do time-travel debugging of generic Python code, but also algorithms that were implemented using Quartz features. I also wrote all the training materials, published internally.
  23. 23. Still at BofA, I developed repository of >50 Computer Science algorithms with visualization. This essentially was a reimplementation of Hotwire for Python. This was implemented using Google AppEngine and cool Python execution. 2013
  24. 24. code test 2016 The goal of Auger was to introspect a running system, record everything, and then produce specific output that uses all the data being recorded. In this case I generate unit tests. But, another application could be to automatically generate performance metrics, or derive types.
  25. 25. Pava Pava takes Java bytecodes and converts them to Python source code. Not sure how useful Pava really is, but for me, this was really an effort to study the differences in bytecodes for Java and Python. 2017
  26. 26. Ikke HappyMac Tempo At Google, I worked a lot on Java and Android tools and apps. I also developed an interest in making the life of engineers better by providing them with tools to understand how they fit into their organisation and how they can profile their life.
  27. 27. 2016-2018 People also represent a complex system and I played a lot with different visualizations of my personal and work graph to discover human connections. github: Ikke Graph.
  28. 28. While working at Uber, my MacBook was always struggling with the CPU usage of Intellij and the Android Emulator. To understand better what was going on, I created HappyMac, which allows its user to discover expensive processes and auto-suspend them using some rules. 2018
  29. 29. HappyWeb is like HappyMac, but focuses on slow network connections. 2018
  30. 30. Tempo constantly watches your MacBook and records the currently active program and saves an event in a local SQL database. The recorded events can show effectiveness of workflows, how much time is spent in certain activities, and how to avoid context switches. A human profiler? 2019
  31. 31. Summary Complex systems are all around us, either as technical solutions or as connections between humans. In order to improve these complex systems, we need to understand them better. To understand them better, we need to measure and generate metrics. Custom tools will then provide insights.
  32. 32. The End “Be humble, we have small brains.”