• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Collective Mind infrastructure and repository to crowdsource auto-tuning (c-mind.org/repo)
 

Collective Mind infrastructure and repository to crowdsource auto-tuning (c-mind.org/repo)

on

  • 173 views

Open access vision publication for this presentation: http://arxiv.org/abs/1308.2410 ...

Open access vision publication for this presentation: http://arxiv.org/abs/1308.2410

Designing, analyzing and optimizing applications for rapidly evolving computer systems is often a tedious, ad-hoc, costly and error prone process due to an enormous number of available design and optimization choices combined with complex interactions between all components. Auto-tuning, run-time adaptation and machine learning based techniques have been investigated for more than a decade to address some of these challenges but are still far from the widespread production use. This is not only due to large optimization spaces, but also due to a lack of a common methodology to discover, preserve and share knowledge about behavior of existing computer systems with ever changing interfaces of analysis and optimization tools.

In this talk I presented a new version of the modular, open source Collective Mind Framework and Repository (cTuning.org, c-mind.org/repo) for collaborative and statistical analysis and optimization of program and architecture behavior. Motivated by physics, biology and AI sciences, this framework helps researchers to gradually expose tuning choices, properties and characteristics at multiple granularity levels in existing systems through multiple plugins. These plugins can be easily combined like LEGO to build customized collaborative or private in-house repositories of shared data (applications, data sets, codelets, micro-benchmarks and architecture descriptions), modules (classification, predictive modeling, run-time adaptation) and statistics from multiple program executions. Collected data is continuously analyzed and extrapolated using online learning to predict better optimizations or hardware configurations to effectively balance performance, power consumption and other characteristics.

This approach was initially validated in the MILEPOST project to remove the training phase of a machine learning based self-tuning compiler, and later extended in the Intel Exascale Lab to connect various tuning tools with an in-house customized repository. During this talk, I will demonstrate the auto-tuning using the new version of this framework and off-the-shelf mobile phones while describing encountered challenges and possible solutions.

Statistics

Views

Total Views
173
Views on SlideShare
173
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Collective Mind infrastructure and repository to crowdsource auto-tuning (c-mind.org/repo) Collective Mind infrastructure and repository to crowdsource auto-tuning (c-mind.org/repo) Presentation Transcript

    • Collective Mind infrastructure and repository to crowdsource auto-tuning (c-mind.org/repo) Grigori Fursin INRIA, France HPSC 2013, Taiwan March 2013
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 2 / 66 • Collective Mind approach combined with social networking, expert knowledge and predictive modeling • Collective Mind framework basics • Plugin-based type-free and schema-free infrastructure • Unified web interfaces (similar to WEB 2.0 concept) • Portable file (json) based repository • Auto-tuning and predictive modeling scenarios • Demo • Conclusions and future works Background
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 3 / 66 Solution Motivation: back to basics Decision (depends on user requirements) Result Available choices (solutions) User requirements: most common: minimize all costs (time, power consumption, price, size, faults, etc) guarantee real-time constraints (bandwidth, QOS, etc) Service/application providers (supercomputing, cloud computing, mobile systems) Should provide choices and help with decisions Hardware and software designers End users Task
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 4 / 66 Solutions Challenges Task Result GCC 4.1.x GCC 4.2.x GCC 4.3.x GCC 4.4.x GCC 4.5.x GCC 4.6.x GCC 4.7.x ICC 10.1 ICC 11.0 ICC 11.1 ICC 12.0 ICC 12.1 LLVM 2.6 LLVM 2.7 LLVM 2.8 LLVM 2.9 LLVM 3.0 Phoenix MVS XLC Open64 Jikes Testarossa OpenMP MPI HMPP OpenCL CUDA gprof prof perf oprofile PAPI TAU Scalasca VTune Amplifierscheduling algorithm- level TBB MKL ATLASprogram- level function- level Codelet loop-level hardware counters IPA polyhedral transformations LTO threadsprocess pass reordering run-time adaptation per phase reconfiguration cache size frequency bandwidth HDD size TLB ISA memory size cores processors threads power consumptionexecution time reliability Clean up this mess! Simplify analysis, tuning and modelling of computer systems for non-computer engineers Bring together researchers from interdisciplinary communities
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 5 / 66 Treat computer system as a black box Task Result Understanding computer systems’ behavior: a physicist’s approach
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 6 / 66 Treat computer system as a black box Task Result Understanding computer systems’ behavior: a physicist’s approach Expose object information flow
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 7 / 66 Treat computer system as a black box Task Result Expose object information flow Object wrapper and repository: observe behavior and keep history information flow expose characteristics Observe system
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 8 / 66 Treat computer system as a black box Task Result Gradually expose properties, characteristics, choices Expose object expose and explore choices information flow Object wrapper and repository: observe behavior and keep history (choices, properties, characteristics, system state, data) information flow expose characteristics Universal Learning Module expose system state expose properties expose characteristics set requirements
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 9 / 66 Treat computer system as a black box Task Result Classify, build models, predict behavior Expose object information flow expose system state Object wrapper and repository: observe behavior and keep history (choices, properties, characteristics, system state, data) history (experience) information flow expose properties expose characteristics expose characteristics continuously build, validate or refine classification and predictive model on the fly expose and explore choices set requirements Universal Learning Module Evolutionary approach, not revolutionary, i.e. do not rebuild existing SW/HW stack from scratch but wrap up existing tools and techniques, and gradually clean up the mess!
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 10 / 66 Unified web interface Unified web interface Unified web interface Unified web interface Expose and exchange optimization knowledge and models in a unified way through http (WEB 2.0) Use distributed, noSQL repository to store highly heterogeneous “big” data Transparently crowdsource learning of a behavior of any existing mobile, cluster, cloud computer system Extrapolate collective knowledge to build faster and more power efficient computer systems Build self-tuning machines using agent-based models
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 11 / 66 Treat computer system as a black box Task Result Gradual decomposition, parameterization, observation and exploration of a system Application Compilers and auxiliary tools Binary and libraries Architecture Run-time environment State of the system Data set Algorithm
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 12 / 66 Treat computer system as a black box Task Result Gradual decomposition, parameterization, observation and exploration of a system Application Compilers and auxiliary tools Binary and libraries Architecture Run-time environment State of the system Data set Algorithm Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models cTuning3 framework aka Collective Mind
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 13 / 66 Treat computer system as a black box Task Result Gradual top-down decomposition, parameterization, observation and exploration of a system Application Compilers and auxiliary tools Binary and libraries Architecture Run-time environment State of the system Data set Algorithm Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Light-weightinterfacetoconnectmodules,dataandmodels cTuning3 framework aka Collective Mind
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 14 / 66 Example of characterizing/explaining behavior of computer systems Gradually expose some characteristics Gradually expose some properties/choices Compile Program time … compiler flags; pragmas … Run code Run-time environment time; CPI, power consumption … pinning/scheduling … System cost; architecture; frequency; cache size… Data set size; values; description … precision … Analyze profile time; size … instrumentation; profiling … Start coarse-grain decomposition of a system (detect coarse-grain effects first). Add universal learning modules. Combine expert knowledge with automatic detection! Start from coarse-grain and gradually move to fine-grain level!
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 15 / 66 0 1 2 3 4 5 6 Program/architecturebehavior:CPI How we can explain the following observations for some piece of code (“codelet object”)? (LU-decomposition codelet, Intel Nehalem) Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 16 / 66 Add 1 property: matrix size 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Program/architecturebehavior:CPI Dataset property: matrix size Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 17 / 66 Try to build a model to correlate objectives (CPI) and features (matrix size). Start from simple models: linear regression (detect coarse grain effects) 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Program/architecturebehavior:CPI Dataset property: matrix size Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 18 / 66 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Program/architecturebehavior:CPI Dataset properties: matrix size If more observations, validate model and detect discrepancies! Continuously retrain models to fit new data! Use model to “focus” exploration on “unusual” behavior! Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 19 / 66 Gradually increase model complexity if needed (hierarchical modeling). For example, detect fine-grain effects (singularities) and characterize them. 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Program/architecturebehavior:CPI Dataset properties: matrix size Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 20 / 66 Start adding more properties (one more architecture with twice bigger cache)! Use automatic approach to correlate all objectives and features. 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Program/architecturebehavior:CPI Dataset properties: matrix size L3 = 4Mb L3 = 8Mb Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 21 / 66 Continuously build and refine classification (decision trees for example) and predictive models on all collected data to improve predictions. Continue exploring design and optimization spaces (evaluate different architectures, optimizations, compilers, etc.) Focus exploration on unexplored areas, areas with high variability or with high mispredict rate of models β εcM predictive model module CPI = ε + 1000 × β × data size Example of characterizing/explaining behavior of computer systems
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 22 / 66 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 1 2 3 4 5 6 Dataset features: matrix size Code/architecturebehavior:CPI Size < 1012 1012 < Size < 2042 Size > 2042 & GCC Size > 2042 & ICC & O2 Size > 2042 & ICC & O3 Optimize decision tree (many different algorithms) Balance precision vs cost of modeling = ROI (coarse-grain vs fine-grain effects) Compact data on-line before sharing with other users! Model optimization and data compaction
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 23 / 66 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Code/architecturebehavior:CPI Dataset features: matrix size Collaboratively and continuously add expert advices or automatic optimizations. Extensible and collaborative advice system
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 24 / 66 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Code/architecturebehavior:CPI Dataset features: matrix size Collaboratively and continuously add expert advices or automatic optimizations. Automatically characterize problem (extract all possible features: hardware counters, semantic features, static features, state of the system, etc) Add manual analysis if needed Extensible and collaborative advice system
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 25 / 66 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Code/architecturebehavior:CPI Dataset features: matrix size cM advice system: Possible problem: Cache conflict misses degrade performance Collaboratively and continuously add expert advices or automatic optimizations. Extensible and collaborative advice system
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 26 / 66 0 1 2 3 4 5 6 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Code/architecturebehavior:CPI Dataset features: matrix size cM advice system: Possible problem: Cache conflict misses degrade performance Possible solution: Array padding (A[N,N] -> A[N,N+1]) Effect: ~30% execution time improvement Collaboratively and continuously add expert advices or automatic optimizations. Extensible and collaborative expert system
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 27 / 66 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Executiontime,sec Dataset features: matrix size Add dynamic memory characterization through semantically non-equivalent modifications. For example, convert all array accesses to scalars to detect balance between CPU/memory accesses. Intentionally change/break semantics to observe reaction in terms of performance/power etc! for for for addr = a[0,0] load … [addr+index1]… mulss … [addr+index2]… subss … [addr+index3]… store … [addr+index4]… addr + = 4 System reaction to code changes: physicist’s view Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 28 / 66 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Executiontime,sec Dataset features: matrix size for for for addr = a[0,0] load … [addr+index1]… mulss … [addr+index2]… subss … [addr+index3]… store … [addr+index4]… addr + = 0 Add dynamic memory characterization through semantically non-equivalent modifications. For example, convert all array accesses to scalars to detect balance between CPU/memory accesses. Intentionally change/break semantics to observe reaction in terms of performance/power etc! Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004 System reaction to code changes: physicist’s view
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 29 / 66 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Executiontime,sec Dataset features: matrix size Expert or automatic advices based on additional information in the repository! Focus optimizations to speed up search: which/where? Advice: Small gap (arithmetic dominates): • Focus on ILP optimizations • Run on complex out-of-order core • Increase processor frequency to speed up application Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004 System reaction to code changes: physicist’s view
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 30 / 66 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Executiontime,sec Dataset features: matrix size Well-knowngapbetweenarithmetic anddataaccesses Advice: Small gap (arithmetic dominates): • Focus on ILP optimizations • Run on complex out-of-order core • Increase processor frequency to speed up application Expert or automatic advices based on additional information in the repository! Focus optimizations to speed up search: which/where? Advice: Big gap (data accesses dominate): • Focus on memory optimizations • Run on simple core • Decrease processor frequency to save power Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004 System reaction to code changes: physicist’s view
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 31 / 66 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Executiontime,sec Dataset features: matrix size Well-knowngapbetweenarithmetic anddataaccesses Advice: Small gap (arithmetic dominates): • Focus on ILP optimizations • Run on complex out-of-order core • Increase processor frequency to speed up application Expert or automatic advices based on additional information in the repository! Focus optimizations to speed up search: which/where? Advice: Big gap (data accesses dominate): • Focus on memory optimizations • Run on simple core • Decrease processor frequency to save power Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004 System reaction to code changes: physicist’s view Can add/remove instructions, Can add/remove threads, etc …
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 32 / 66 cd [application_directory] make CC=icc CC_OPTS=-fast or icc -fast *.c time ./a.out < [my_dataset] > [output] record “-fast”, execution time Implementation in open-source cTuning1 framework
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 33 / 66 Implementation in open-source cTuning1 framework cd [application_directory] make CC=icc CC_OPTS=-fast or icc -fast *.c time ./a.out < [my_dataset] > [output] record “-fast”, execution time ccc-comp build=“make” compiler=icc opts=“-fast” ccc-comp compiler=icc opts=“-fast” ccc-run prog=./a.out cmd=“< [my dataset]” ccc-time <cmd> ccc-record-stats <file_with_stats> • Low level platform-dependent plugins in C • Communication through text file or directly through MySQL database • High level platform-independent exploration or analysis plugins in PHP • Web services at cTuning.org as plugins in PHP
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 34 / 66 Compilation abstraction supports multiple compilers and languages ccc-comp Compiler(s) Multiple user architectures Execution abstraction supports multiple architectures, remote SSH execution, profiling ccc-run Low-level execution abstraction timing, profiling, hardware counters ccc-time Application Interactive Compilation Interface fine-grain analysis and optimization Multiple datasets (from cBench) Implementation in open-source cTuning1 framework
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 35 / 66 High-level optimization tools compiler and platform independent - perform iterative optimization: compiler flags or ICI passes/parameters [random, one off, one by one, focused] - user optimization plugins ccc-run-* Compilation abstraction supports multiple compilers and languages ccc-comp Compiler(s) Multiple user architectures High-level machine learning tools - build ML models - extract program static/dynamic features - predict “good” program optimizations ccc-ml-* Execution abstraction supports multiple architectures, remote SSH execution, profiling ccc-run Low-level execution abstraction timing, profiling, hardware counters ccc-time Application Interactive Compilation Interface fine-grain analysis and optimization Multiple datasets (from cBench) Implementation in open-source cTuning1 framework
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 36 / 66 High-level optimization tools compiler and platform independent - perform iterative optimization: compiler flags or ICI passes/parameters [random, one off, one by one, focused] - user optimization plugins ccc-run-* Compilation abstraction supports multiple compilers and languages ccc-comp Communication with COD ccc-db-send-stats-* Compiler(s) Multiple user architectures Collective Optimization Database (COD) MySQL Common database: keep info about all architectures, environments, programs, compilers, etc Optimization database: keep info about interesting optimization cases from the community and expert advices/knowledge High-level machine learning tools - build ML models - extract program static/dynamic features - predict “good” program optimizations ccc-ml-* COD tools - administration db/admin - optimization analysis db/analysis Execution abstraction supports multiple architectures, remote SSH execution, profiling ccc-run Communication with COD ccc-db-send-stats* Low-level execution abstraction timing, profiling, hardware counters ccc-time Application Interactive Compilation Interface fine-grain analysis and optimization Multiple datasets (from cBench) CCC configuration configure all tools, COD, architecture, compiler, benchmarks, etc ccc-configure-* Implementation in open-source cTuning1 framework
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 37 / 66 High-level optimization tools compiler and platform independent - perform iterative optimization: compiler flags or ICI passes/parameters [random, one off, one by one, focused] - user optimization plugins ccc-run-* Compilation abstraction supports multiple compilers and languages ccc-comp Communication with COD ccc-db-send-stats-* Compiler(s) Multiple user architectures Collective Optimization Database (COD) MySQL Common database: keep info about all architectures, environments, programs, compilers, etc Optimization database: keep info about interesting optimization cases from the community and expert advices/knowledge High-level machine learning tools - build ML models - extract program static/dynamic features - predict “good” program optimizations ccc-ml-* COD tools - administration db/admin - optimization analysis db/analysis Execution abstraction supports multiple architectures, remote SSH execution, profiling ccc-run Communication with COD ccc-db-send-stats* Low-level execution abstraction timing, profiling, hardware counters ccc-time Application Interactive Compilation Interface fine-grain analysis and optimization Multiple datasets (from cBench) Web access to COD http://ctuning.org CCC configuration configure all tools, COD, architecture, compiler, benchmarks, etc ccc-configure-* Implementation in open-source cTuning1 framework
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 38 / 66 MySQL-based Collective Optimization Database Platforms unique PLATFORM_ID Compilers unique COMPILER_ID Runtime environments unique RE_ID Programs unique PROGRAM_ID Datasets unique DATASET_ID Platform features unique PLATFORM_FEATURE_ID Global platform optimization flags unique OPT_PLATFORM_ID Global optimization flags unique OPT_ID Optimization passes unique OPT_PASSES_ID Compilation info unique COMPILE_ID Execution info unique RUN_ID unique RUN_ID_ASSOCIATE Program passes associated COMPILE_ID Program features associated COMPILE_ID Common Optimization Database (shared among all users) Local or shared databases with optimization cases
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 39 / 66 Problems with cTuning1 • Difficult to extend (C, various hardwired components, need to change schema and types in MySQL) • No convenient way of sharing modules, benchmarks, data sets, models (manual, csv files, emails, etc) • Problems with repository scalability • Complex, hardwired interfaces
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 40 / 66 cd [application_directory] make CC=icc CC_OPTS=-fast or icc -fast *.c time ./a.out < [my_dataset] > [output] record “-fast”, execution time cTuning3 aka Collective Mind framework basics
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 41 / 66 cd [application_directory] make CC=icc CC_OPTS=-fast or icc -fast *.c time ./a.out < [my_dataset] > [output] record “-fast”, execution time cTuning3 aka Collective Mind framework basics cM kernel code.source build Universal cM FE cM plugins (modules) code run compiler build … E nd-users or cM developers CMD python python or any other language
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 42 / 66 cd [application_directory] make CC=icc CC_OPTS=-fast or icc -fast *.c time ./a.out < [my_dataset] > [output] record “-fast”, execution time cTuning3 aka Collective Mind framework basics cM kernel code.source build Universal cM FE cM plugins (modules) code run compiler build … E nd-users or cM developers CMD python python or any other language cm [module name] [action] (param1=value1 param2=value2 … -- unparsed command line) cm code.source build ct_compiler=icc13 ct_optimizations=-fast cm compiler build -- icc -fast *.c cm code run os=android binary=./a.out dataset=image-crazy-scientist.pgm Should be able to run on any OS (Windows, Linux, Android, MacOS, etc)!
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 43 / 66 cTuning3 aka Collective Mind framework basics Simple and minimalistic high-level cM interface - one function (!) should be easy to connect to any language if needed schema and type-free (only strings) - easily extended when needed for research (agile methodology)! (python dictionary) output = cm_kernel.access ( (python dictionary) input ) Input: { cm_run_module_uoa - cM plugin name (or some UID) cm_action - cM plugin action (function) parameters - (module and action dependent) } Output: { cm_return - if 0, success if >0, error if <0, warning cm_error - if cm_return>0, error message parameters - (module and action dependent) }
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 44 / 66 Application Compilers and auxiliary tools Binary and libraries Architecture Run-time environment State of the system Data set Algorithm Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Collective Mind Repository basics
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 45 / 66 Application Compilers and auxiliary tools Binary and libraries Architecture Run-time environment State of the system Data set Algorithm Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models Repo/models .cmr / module UID or alias (cM UOA) / data UID or alias (cm UOA) Repository root First level directory Second level directory Very flexible and portable Easy to access, edit and move data Can be per application, experiment, architecture, etc Can be easily shared (through web, SVN, GIT, FTP) Collective Mind Repository basics
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 46 / 66 Schema-free extensible data meta-representation cM uses JSON as internal data representation format: JSON or JavaScript Object Notation, is a text-based open standard designed for human-readable data interchange (from Wikipedia) • very intuitive to use and modify • nearly native for python and php; simple libraries for Java, C, C++, … • easy to index with powerful indexing services (cM uses ElasticSearch) cM records input and output of the module for reproducibility! Data is referenced by CID: (Repository UID:) Module UID: Data UID
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 47 / 66 Example of JSON entry ctuning.compiler:icc-12.x-linux { "all_compiler_flags_desc": { "##base_flag": { "type": "text“ "desc_text": "compiler flag: -O3", "field_size": "7", "has_choice": "yes", "choice": [ "-O0", "-O1", "-O2", "-Os", "-O3", "-fast" ], "default_value": "-O3", "explorable": "yes", "explore_level": "1", "explore_type": "fixed", "forbid_disable_at_random": "yes" }, … } … } Schema-free extensible data representation
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 48 / 66 Treat computer system as a black box Task Result Execute code Input dictionary that can be represented as JSON {“run_cmd”:’image’, “binary_name”:”a.out”, “run_set_env”:[], “run_output_files”:[], “work_dir”:””, “run_time_out”:””, “code_deps”:[], “run_os_uoa”:””, …} Function (for example “run”) invoke object, observe behavior and keep history python module that has a UID and alias (for example, “code” / 688dee2e7014f1fb) Universal modules/functions Output dictionary that can be represented as JSON {“cm_return”:’’, “failed_run”:””, “run_time_by_module”:””, …} gradually describe input and output (choices, characteristics, etc) Save input/output and related files Index input and output for fast search
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 49 / 66 cM repository (.cmr) cM data Web Command line and scripts cM kernel Programs Tools Systems OpenME cM interface php shell (cm) C C++ Fortran Java End user research and development scenarios End user instrumented and adaptive programs, tools and systems cM kernel in Python* •native support for other languages maybe added later access load data load module create entry delete entry list entries auth user … kernel core repo user module web code code.source os package dataset … … … … … … … … … … … … Collective Mind overall structure • Gradually add more modules, interfaces and data depending on user/project/company needs • Gradually add more parameters • Gradually expose choices, properties, characteristics Community or workgroup cmx - user- friendly CMD extension
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 50 / 66 Academia: public, open-source modules and data Collaborative, reproducible experiments: research LEGO code.source code system os dataset package ctuning.processor ctuning.compiler choice os.script Industry: proprietary modules and data code.source code system os dataset package ctuning.processor ctuning.compiler choice os.script •Continuously adding “basic blocks” (modules) •Adding tools, applications, datasets •Gradually stabilize interfaces Users can start connecting modules and data together to prepare experimental pipelines with various observation, characterization, auto-tuning and predictive scenarios!
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 51 / 66 Experimental pipelines for auto-tuning and modeling •Init pipeline •Detected system information •Initialize parameters •Prepare dataset •Clean program •Prepare compiler flags •Use compiler profiling •Use cTuning CC/MILEPOST GCC for fine-grain program analysis and tuning •Use universal Alchemist plugin (with any OpenME-compatible compiler or tool) •Use Alchemist plugin (currently for GCC) •Build program •Get objdump and md5sum (if supported) •Use OpenME for fine-grain program analysis and online tuning (build & run) •Use 'Intel VTune Amplifier' to collect hardware counters •Use 'perf' to collect hardware counters •Set frequency (in Unix, if supported) •Get system state before execution •Run program •Check output for correctness (use dataset UID to save different outputs) •Finish OpenME •Misc info •Observed characteristics •Observed statistical characteristics •Finalize pipeline
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 52 / 66 Currently prepared experiments •Polybench - numerical kernels with exposed parameters of all matrices in cM • CPU: 28 prepared benchmarks • CUDA: 15 prepared benchmarks • OpenCL: 15 prepared benchmarks • cBench - 23 benchmarks with 20 and 1000 datasets per benchmark • Codelets - 44 codelets from embedded domain (provided by CAPS Entreprise) • SPEC 2000/2006 • Description of 32-bit and 64-bit OS: Windows, Linux, Android • Description of major compilers: GCC 4.x, LLVM 3.x, Open64/Pathscale 5.x, ICC 12.x • Support for collection of hardware counters: perf, Intel vTune • Support for frequency modification • Validated on laptops, mobiles, tables, GRID/cloud - can work even from the USB key
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 53 / 66 Visualize and analyze optimization spaces Program: cBench: susan corners Processor: ARM v6, 830MHz Compiler: Sourcery GCC for ARM v4.6.1 OS: Android OS v2.3.5 System: Samsung Galaxy Y Data set: MiDataSet #1, image, 600x450x8b PGM, 263KB
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 54 / 66 Gradually increase granularity and complexity Gradually expose some characteristics Gradually expose some choices Algorithm selection (time) productivity, variable- accuracy, complexity … Language, MPI, OpenMP, TBB, MapReduce … Compile Program time … compiler flags; pragmas … Code analysis & Transformations time; memory usage; code size … transformation ordering; polyhedral transformations; transformation parameters; instruction ordering … Process Thread Function Codelet Loop Instruction Run code Run-time environment time; power consumption … pinning/scheduling … System cost; size … CPU/GPU; frequency; memory hierarchy … Data set size; values; description … precision … Run-time analysis time; precision … hardware counters; power meters … Run-time state processor state; cache state … helper threads; hardware counters … Analyze profile time; size … instrumentation; profiling … Coarse-grain vs. fine-grain effects: depends on user requirements and expected ROI
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 55 / 66 Interactive compilers, tools and applications Application Source-to-source transformation tools binary execution Binary transformation tools Production Compilers Traditional compilation, analysis and optimization Often internal compiler decisions are not known or there is no precise control even through pragmas. Interference with internal compiler optimizations complicates program analysis and characterization. Current pragma based auto- tuning frameworks are very complex. What about finer- grain level?
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 56 / 66 Detect optimization flags Optimization manager Pass 1 GCC Data Layer AST, CFG, CF, etc Compiler ... Pass N OpenME - interactive plugin and event-based interface to “open up” applications and tools
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 57 / 66 Detect optimization flags Optimization manager Event Pass N Event Pass 1 GCC Data Layer AST, CFG, CF, etc Feature Event Open ME Compiler with OpenME ... OpenME - interactive plugin and event-based interface to “open up” applications and tools
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 58 / 66 Detect optimization flags Optimization manager Event Pass N Event Pass 1 GCC Data Layer AST, CFG, CF, etc Feature Event Open ME ... OpenME Plugins cM modules Selecting pass sequences Extracting static program features <Dynamically linked shared libraries> ... OpenME - interactive plugin and event-based interface to “open up” applications and tools Compiler with OpenME
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 59 / 66 Detect optimization flags Optimization manager Event Pass N Event Pass 1 GCC Data Layer AST, CFG, CF, etc Feature Event Open ME ... OpenME Plugins cM modules Selecting pass sequences Extracting static program features <Dynamically linked shared libraries> ... We collaborated with Google and Mozilla to move similar framework to mainline GCC so that everyone can use it for research. Now available in GCC >=4.6 OpenME - interactive plugin and event-based interface to “open up” applications and tools Compiler with OpenME
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 60 / 66 Add cM wrapper (compiler) Application Source-to-source transformation tools Production Compiler with OpenME binary execution Binary transformation tools Very simple plugin framework for any compiler or tool Full control over optimization decisions! Remove interference between different tools cM framework OpenME - interactive plugin and event-based interface to “open up” applications and tools
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 61 / 66 Example of OpenME for LLVM 3.2 tools/clang/tools/driver/cc1_main.cpp #include "openme.h“ … int cc1_main(const char **ArgBegin, const char **ArgEnd, const char *Argv0, void *MainAddr) { openme_init("UNI_ALCHEMIST_USE", "UNI_ALCHEMIST_PLUGINS", NULL, 0); … // Execute the frontend actions. Success = ExecuteCompilerInvocation(Clang.get()); openme_callback("ALC_FINISH", NULL); … } OpenME: 3 functions only! • openme_init(…) - initialize/load plugin • openme_callback(char* event_name, void* params) - call event • openme_finish(…) - finalize (if needed)
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 62 / 66 Example of OpenME for LLVM 3.2 lib/Transforms/Scalar/LoopUnrollPass.cpp #include <cJSON.h> #include "openme.h“ … bool LoopUnroll::runOnLoop(Loop *L, LPPassManager &LPM) { struct alc_unroll { const char *func_name; const char *loop_name; cJSON *json; int factor; } alc_unroll; … alc_unroll.func_name=(Header->getParent()->getName()).data(); alc_unroll.loop_name=(Header->getName()).data(); openme_callback("ALC_TRANSFORM_UNROLL_INIT", &alc_unroll); … // Unroll the loop. alc_unroll.factor=Count; openme_callback("ALC_TRANSFORM_UNROLL", &alc_unroll); Count=alc_unroll.factor; if (!UnrollLoop(L, Count, TripCount, UnrollRuntime, TripMultiple, LI, &LPM)) return false; … }
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 63 / 66 Example of OpenME for LLVM 3.2 Alchemist plugin (.so/dll object) - in development for online/interactive analysis, tuning and adaptation #include <cJSON.h> #include <openme.h> int openme_plugin_init(struct openme_info *ome_info) { … openme_register_callback(ome_info, "ALC_TRANSFORM_UNROLL_INIT", alc_transform_unroll_init); openme_register_callback(ome_info, "ALC_TRANSFORM_UNROLL", alc_transform_unroll); openme_register_callback(ome_info, "ALC_TRANSFORM_UNROLL_FEATURES", alc_transform_unroll_features); openme_register_callback(ome_info, "ALC_FINISH", alc_finish); … } extern void alc_transform_unroll_init(struct alc_unroll *alc_unroll){ … } extern void alc_transform_unroll(struct alc_unroll *alc_unroll) { … } …
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 64 / 66 Example of OpenME for OpenCL/CUDA C application • 2mm.c / 2mm.cu … #ifdef OPENME #include <openme.h> #endif … int main(void) { … #ifdef OPENME openme_init(NULL,NULL,NULL,0); openme_callback("PROGRAM_START", NULL); #endif … #ifdef OPENME openme_callback("ACC_KERNEL_START", NULL); #endif cl_launch_kernel(); or mm2Cuda(A, B, C, D, E, E_outputFromGpu); #ifdef OPENME openme_callback("ACC_KERNEL_END", NULL); #endif … … #ifdef OPENME openme_callback("KERNEL_START", NULL); #endif mm2_cpu(A, B, C, D, E); #ifdef OPENME openme_callback("KERNEL_END", NULL); #endif #ifdef OPENME openme_callback("PROGRAM_END", NULL); #endif … }
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 65 / 66 Example of OpenME for Fortran application • matmul.F PROGRAM MATMULPROG … INTEGER*8 OBJ, OPENME_CREATE_OBJ_F CALL OPENME_INIT_F(""//CHAR(0), ""//CHAR(0), ""//CHAR(0), 0) CALL OPENME_CALLBACK_F("PROGRAM_START"//CHAR(0)) … CALL OPENME_CALLBACK_F("KERNEL_START"//CHAR(0)); DO I=1, I_REPEAT CALL MATMUL END DO CALL OPENME_CALLBACK_F("KERNEL_END"//CHAR(0)); … CALL OPENME_CALLBACK_F("PROGRAM_END"//CHAR(0)) END
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 66 / 66 1) Prepare pre-release around May/June 2013 (BSD-style license) - ASK for preview! 2) Reproduce my past published research within new framework: • Add “classical” classification and predictive models • Add various exploration strategies (random, focused) • Add run-time adaptation scenarios (CUDA/OpenCL scheduling, pinning, etc) • Add co-design scenarios 3) Use framework for analysis and auto-tuning of industrial applications 4) Help to customize framework for industrial usages (consulting) 5) Applying for new funding (academic and industrial) 6) Continue virtual collaborative cTuning Lab to build community: • Public repository to share applications, datasets, models at cTuning.org: • New publication model for reproducible research • Community R&D discussion http://groups.google.com/group/collective-mind • Collect data from Android mobiles Next steps
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 67 / 66 Acknowledgements • PhD students and postdocs (my Intel Exascale team) Abdul Wahid Memon, Pablo Oliveira, Yuriy Kashnikov • Colleague from NCAR, USA Davide Del Vento and his colleagues/interns • Colleagues from IBM, CAPS, ARC (Synopsys), Intel, Google, ARM, ST • Colleagues from Intel (USA) David Kuck and David Wong • cTuning community: • EU FP6, FP7 program and HiPEAC network of excellence http://www.hipeac.net
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 68 / 66 Main references • Grigori Fursin. Collective Tuning Initiative: automating and accelerating development and optimization of computing systems. Proceedings of the GCC Summit’09, Montreal, Canada, June 2009 • Grigori Fursin and Olivier Temam. Collective Optimization: A Practical Collaborative Approach. ACM Transactions on Architecture and Code Optimization (TACO), December 2010, Volume 7, Number 4, pages 20-49 • Grigori Fursin, Yuriy Kashnikov, Abdul Wahid Memon, Zbigniew Chamski, Olivier Temam, Mircea Namolaru, Elad Yom-Tov, Bilha Mendelson, Ayal Zaks, Eric Courtois, Francois Bodin, Phil Barnard, Elton Ashton, Edwin Bonilla, John Thomson, Chris Williams, Michael O'Boyle. MILEPOST GCC: machine learning enabled self-tuning compiler. International Journal of Parallel Programming (IJPP), June 2011, Volume 39, Issue 3, pages 296-327 • Yang Chen, Shuangde Fang, Yuanjie Huang, Lieven Eeckhout, Grigori Fursin, Olivier Temam and Chengyong Wu. Deconstructing iterative optimization. ACM Transactions on Architecture and Code Optimization (TACO), October 2012, Volume 9, Number 3 • Yang Chen, Yuanjie Huang, Lieven Eeckhout, Grigori Fursin, Liang Peng, Olivier Temam, Chengyong Wu. Evaluating Iterative Optimization across 1000 Data Sets. PLDI'10 • Victor Jimenez, Isaac Gelado, Lluis Vilanova, Marisa Gil, Grigori Fursin and Nacho Navarro. Predictive runtime code scheduling for heterogeneous architectures. HiPEAC’09
    • Grigori Fursin “Collective Mind: novel methodology, framework and repository to crowdsource auto-tuning” HPSC 2013, NTU, Taiwan March, 2013 69 / 66 Main references • Lianjie Luo, Yang Chen, Chengyong Wu, Shun Long and Grigori Fursin. Finding representative sets of optimizations for adaptive multiversioning applications. SMART'09 co-located with HiPEAC'09 • Grigori Fursin, John Cavazos, Michael O'Boyle and Olivier Temam. MiDataSets: Creating The Conditions For A More Realistic Evaluation of Iterative Optimization. HiPEAC’07 • F. Agakov, E. Bonilla, J. Cavazos, B. Franke, G. Fursin, M.F.P. O'Boyle, J. Thomson, M. Toussaint and C.K.I. Williams. Using Machine Learning to Focus Iterative Optimization. CGO’06 •Grigori Fursin, Albert Cohen, Michael O'Boyle and Oliver Temam. A Practical Method For Quickly Evaluating Program Optimizations. HiPEAC’05 •Grigori Fursin, Mike O'Boyle, Olivier Temam, and Gregory Watts. Fast and Accurate Method for Determining a Lower Bound on Execution Time. Concurrency Practice and Experience, 16(2-3), pages 271-292, 2004 • Grigori Fursin. Iterative Compilation and Performance Prediction for Numerical Applications. Ph.D. thesis, University of Edinburgh, Edinburgh, UK, January 2004