• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Speeding Up the Evaluation of Evolutionary Learning Systems Using GPGPUs
 

Speeding Up the Evaluation of Evolutionary Learning Systems Using GPGPUs

on

  • 1,103 views

Presentation given at GECCO 2010 in Portland, Oregon. Winner of the Genetic Based Machine Learning track Best Paper Award.

Presentation given at GECCO 2010 in Portland, Oregon. Winner of the Genetic Based Machine Learning track Best Paper Award.

Statistics

Views

Total Views
1,103
Views on SlideShare
1,094
Embed Views
9

Actions

Likes
1
Downloads
17
Comments
0

2 Embeds 9

http://www.cs.nott.ac.uk 7
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Speeding Up the Evaluation of Evolutionary Learning Systems Using GPGPUs Speeding Up the Evaluation of Evolutionary Learning Systems Using GPGPUs Presentation Transcript

    • BioHEL GBML System BioHEL using CUDA Experiments and results Conclusions and Further Work Speeding up the Evaluation of Evolutionary Learning Systems using GPGPUs María A. Franco, Natalio Krasnogor and Jaume Bacardit University of Nottingham, UK, ASAP Research Group, School of Computer Science {mxf,nxk,jqb}@cs.nott.ac.uk July 10, 2010 M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 1 / 27
    • BioHEL GBML System BioHEL using CUDA Experiments and results Conclusions and Further Work Motivation Nowadays the data collection rate easily exceeds the processing/data-mining rate. Real-life problems = big + complex. There is a need to improve the efficiency of evolutionary learning systems to cope with large scale domains[Sastry, 2005][Bacardit and Llorà, 2009]. This work is focused on boosting the performance of the BioHEL system by using the processing capacity inside GPGPUs. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 2 / 27
    • BioHEL GBML System BioHEL using CUDA Experiments and results Conclusions and Further Work Outline 1 BioHEL BioHEL GBML System Characteristics of BioHEL 2 BioHEL using CUDA How does CUDA works? Challenges of using CUDA Implementation details 3 Experiments and results Stage 1: Raw evaluation Stage 2: Integration with BioHEL 4 Conclusions and Further Work M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 3 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work The BioHEL GBML System BIOinformatics-oriented Hierarchical Evolutionary Learning - BioHEL[Bacardit et al., 2009] BioHEL was designed to handle large scale bioinformatics datasets[Stout et al., 2008] BioHEL is a GBML system that employs the Iterative Rule Learning (IRL) paradigm First used in EC in Venturini’s SIA system[Venturini, 1993] Widely used for both Fuzzy and non-fuzzy evolutionary learning M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 4 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work The BioHEL GBML System BIOinformatics-oriented Hierarchical Evolutionary Learning - BioHEL[Bacardit et al., 2009] BioHEL was designed to handle large scale bioinformatics datasets[Stout et al., 2008] BioHEL is a GBML system that employs the Iterative Rule Learning (IRL) paradigm First used in EC in Venturini’s SIA system[Venturini, 1993] Widely used for both Fuzzy and non-fuzzy evolutionary learning M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 4 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work The BioHEL GBML System BIOinformatics-oriented Hierarchical Evolutionary Learning - BioHEL[Bacardit et al., 2009] BioHEL was designed to handle large scale bioinformatics datasets[Stout et al., 2008] BioHEL is a GBML system that employs the Iterative Rule Learning (IRL) paradigm First used in EC in Venturini’s SIA system[Venturini, 1993] Widely used for both Fuzzy and non-fuzzy evolutionary learning M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 4 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work Characteristics of BioHEL The fitness function based on the Minimum-Description-Length (MDL)[Rissanen, 1978] principle that tries to Evolve accurate rules Evolve high coverage rules Evolve rules with low complexity, as general as possible BioHEL applies a supervised learning paradigm. To compute the fitness we need three metrics per classifier computed from the training set: 1 # of instances that match the condition of the rule 2 # of instances that match the action of the rule 3 # of instances that match the condition and the action M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 5 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work Characteristics of BioHEL The fitness function based on the Minimum-Description-Length (MDL)[Rissanen, 1978] principle that tries to Evolve accurate rules Evolve high coverage rules Evolve rules with low complexity, as general as possible BioHEL applies a supervised learning paradigm. To compute the fitness we need three metrics per classifier computed from the training set: 1 # of instances that match the condition of the rule 2 # of instances that match the action of the rule 3 # of instances that match the condition and the action M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 5 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work Characteristics of BioHEL The ILAS windowing scheme[Bacardit, 2004] Efficiency enhancement method, the training set is divided in non-overlapping strata and each iteration uses a different strata for its fitness calculations. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 6 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work Evaluation process in BioHEL The computationally heavy stage of the evaluation process is the match process. We perform stages 1 and 2 inside the GPGPU. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 7 / 27
    • BioHEL GBML System BioHEL using CUDA BioHEL GBML System Experiments and results Characteristics of BioHEL Conclusions and Further Work Evaluation process in BioHEL The computationally heavy stage of the evaluation process is the match process. We perform stages 1 and 2 inside the GPGPU. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 7 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Why using GPGPUs? This GPGPUs acceleration have been used already in GP[Langdon and Harrison, 2008], GAs[Maitre et al., 2009] and LCS[Loiacono and Lanzi, 2009] The use of GPGPUs in Machine Learning involves a greater challenge because it deals with very large volumes of data However, this also means that it is potentially more parallelizable. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 8 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Why using GPGPUs? This GPGPUs acceleration have been used already in GP[Langdon and Harrison, 2008], GAs[Maitre et al., 2009] and LCS[Loiacono and Lanzi, 2009] The use of GPGPUs in Machine Learning involves a greater challenge because it deals with very large volumes of data However, this also means that it is potentially more parallelizable. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 8 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Why using GPGPUs? This GPGPUs acceleration have been used already in GP[Langdon and Harrison, 2008], GAs[Maitre et al., 2009] and LCS[Loiacono and Lanzi, 2009] The use of GPGPUs in Machine Learning involves a greater challenge because it deals with very large volumes of data However, this also means that it is potentially more parallelizable. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 8 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Architecture NVIDIA Computed Unified Device Architecture (CUDA) is a parallel computing architecture that exploits the capacity within NVIDIA’s Graphic Processor Units. CUDA runs thousands of threads at the same time ⇒ SPMD (Same Program Multiple Data) paradigm M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 9 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Architecture NVIDIA Computed Unified Device Architecture (CUDA) is a parallel computing architecture that exploits the capacity within NVIDIA’s Graphic Processor Units. CUDA runs thousands of threads at the same time ⇒ SPMD (Same Program Multiple Data) paradigm M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 9 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Architecture M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 10 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Memories Different types of memory with different access speed. The memory is limited The memory copy operations involve a considerable amount of execution time Since we are aiming to work with large scale datasets a good strategy to minimize the execution time is based on the memory usage M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 11 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Memories Different types of memory with different access speed. The memory is limited The memory copy operations involve a considerable amount of execution time Since we are aiming to work with large scale datasets a good strategy to minimize the execution time is based on the memory usage M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 11 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work CUDA Memories Different types of memory with different access speed. The memory is limited The memory copy operations involve a considerable amount of execution time Since we are aiming to work with large scale datasets a good strategy to minimize the execution time is based on the memory usage M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 11 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work The ideal way would be... Copy all the classifiers and instances and launch one thread for each classifier-instance comparison M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 12 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Challenges of using CUDA Problem 1: Memory copy operations In each iteration the classifiers and the instances to compare are different. They need to be copied again into global memory in each iteration. Problem 2: Memory bounds It might be not possible to store all the classifiers and the example instances to make all the comparisons at the same time. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 13 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Challenges of using CUDA Problem 1: Memory copy operations In each iteration the classifiers and the instances to compare are different. They need to be copied again into global memory in each iteration. Problem 2: Memory bounds It might be not possible to store all the classifiers and the example instances to make all the comparisons at the same time. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 13 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Solution: Memory calculations If all the instances fit in memory... We copy the instances only one time at the beginning of each GA run and access different windows by using a memory offset. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 14 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Solution: Memory calculations If all the instances fit in memory... We copy the instances only one time at the beginning of each GA run and access different windows by using a memory offset. If all the instances do not fit in memory... We calculate the number of classifiers and instances that fit in memory in order to minimise the memory copy operations. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 14 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Challenges of using CUDA Problem 3: Output structure size If we make N × M comparisons we will have to copy back to host a structure of size O(N × M) which is very slow Solution: Compute the total in device memory M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 15 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Challenges of using CUDA Problem 3: Output structure size If we make N × M comparisons we will have to copy back to host a structure of size O(N × M) which is very slow Solution: Compute the total in device memory Reduce the three values in device memory in other to minimize the time spend in memory copy operations. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 15 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Kernel functions M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 16 / 27
    • BioHEL GBML System How does CUDA works? BioHEL using CUDA Challenges of using CUDA Experiments and results Implementation details Conclusions and Further Work Kernel functions M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 16 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Experiments Set up Two stages of experiments Evaluation process independently Integration with the learning process Different functions to manage discrete, continuous and mixed problems We check the integration with the ILAS windowing system Cuda Experiments Serial Experiments Pentium 4 of 3.6GHz, 2GB HPC facility and the UoN RAM and a Tesla C1060 with each node with 2 quad-core 4GB of global memory and processors (Intel Xeon 30 multiprocessors E5472 3.0GHz) M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 17 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Experiments Set up Two stages of experiments Evaluation process independently Integration with the learning process Different functions to manage discrete, continuous and mixed problems We check the integration with the ILAS windowing system Cuda Experiments Serial Experiments Pentium 4 of 3.6GHz, 2GB HPC facility and the UoN RAM and a Tesla C1060 with each node with 2 quad-core 4GB of global memory and processors (Intel Xeon 30 multiprocessors E5472 3.0GHz) M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 17 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Speed in the evaluation process Name |T| #Att #Disc #Cont #Cl T. Serial (s) T.CUDA (s) Speed Up sat 5790 36 0 36 6 3.60± 0.21 1.92±0.01 1.9 wav 4539 40 0 40 3 2.57± 0.08 1.59±0.01 1.6 Cont. pen 9892 16 0 16 10 4.94± 0.24 2.25±0.02 2.2 SS 75583 300 0 300 3 770.61± 119.49 14.69±0.23 52.4 CN 234638 180 0 180 2 1555.90± 452.79 42.35±0.55 36.7 adu 43960 14 8 6 2 147.86± 30.93 10.38±0.09 14.2 far 90868 29 24 5 8 420.78± 90.58 23.13±1.04 18.2 kdd 444619 41 15 26 23 1715.66± 632.40 95.89±1.42 17.9 Mixed SA 493788 270 26 244 2 3776.36±1212.84 90.45±1.17 41.8 Par 235929 18 18 0 2 863.72± 163.13 60.04±0.58 14.4 c-4 60803 42 42 0 3 343.75± 71.93 17.86±0.18 19.2 M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 18 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Speed Up vs. Training set size Speed Up according to the training set size 60 adu - 14atts pen - 16atts Par - 18atts far - 29atts 50 sat - 36atts wav - 40atts kdd - 41atts c-4 - 42atts CN - 180atts 40 SA - 270atts SS - 300atts Speed Up 30 20 10 0 100 1000 10000 100000 1e+06 Training set size M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 19 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Integration with the ILAS Windowing scheme Total Speedup According to the Number of Windows 700 adu - 14atts pen - 16atts Par - 18atts 600 far - 29atts sat - 36atts wav - 40atts kdd - 41atts c-4 - 42atts 500 CN - 180atts SA - 270atts SS - 300atts 400 Speed Up 300 200 100 0 5 10 15 20 25 30 35 40 45 50 Number of Windows M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 20 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work How do we get speed up? The continuous problems get more speed up than the mixed problems. The problems with large number of attributes or large training sets get more speed up. There is a sweet-spot were the combination between the CUDA fitness function and the ILAS Windowing scheme produces more speedup. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 21 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Speed up of BioHEL using CUDA Name |T| #Att #Disc #Cont #Cl T. Serial (s) T.CUDA (s) Speed Up sat 5790 36 0 36 6 0.03± 0.01 25.91± 2.45 3.7 wav 4539 40 0 40 3 75.47± 9.38 24.69± 0.81 3.1 Cont. pen 9892 16 0 16 10 149.70± 19.93 40.04± 2.94 3.7 SS 75583 300 0 300 3 347979.80± 60982.74 5992.28±247.50 58.1 CN 234638 180 0 180 2 821464.70±167542.04 18644.31±943.98 44.1 adu 43960 14 8 6 2 5422.78± 1410.71 271.73± 26.03 20.0 far 90868 29 24 5 8 2471.28± 701.83 94.99± 41.53 26.0 kdd 444619 41 15 26 23 76442.32± 23533.21 2102.414±191.34 36.4 Mixed SA 493788 270 26 244 2 1252976.80±203186.55 28759.71±552.00 38.3 Par 235929 18 18 0 2 524706.70± 98949.46 19559.79±671.70 26.8 c-4 60803 42 42 0 3 52917.95± 8059.55 2417.83±170.19 21.9 M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 22 / 27
    • BioHEL GBML System Set up BioHEL using CUDA Stage 1: Raw evaluation Experiments and results Stage 2: Integration with BioHEL Conclusions and Further Work Speed up of BioHEL using CUDA Name |T| #Att #Disc #Cont #Cl T. Serial (s) T.CUDA (s) Speed Up sat 5790 36 0 36 6 0.03± 0.01 25.91± 2.45 3.7 wav 4539 40 0 40 3 75.47± 9.38 24.69± 0.81 3.1 Cont. pen 9892 16 0 16 10 149.70± 19.93 40.04± 2.94 3.7 SS 75583 300 0 300 3 347979.80± 60982.74 5992.28±247.50 58.1 CN 234638 180 0 180 2 821464.70±167542.04 18644.31±943.98 44.1 adu 43960 14 8 6 2 5422.78± 1410.71 271.73± 26.03 20.0 far 90868 29 24 5 8 2471.28± 701.83 94.99± 41.53 26.0 kdd 444619 41 15 26 23 76442.32± 23533.21 2102.414±191.34 36.4 Mixed SA 493788 270 26 244 2 1252976.80±203186.55 28759.71±552.00 38.3 Par 235929 18 18 0 2 524706.70± 98949.46 19559.79±671.70 26.8 c-4 60803 42 42 0 3 52917.95± 8059.55 2417.83±170.19 21.9 The experiments that took 2 weeks to finish now run in 8 hours! M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 22 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Conclusions CUDA allows us to exploit the intrinsic parallelism within the populations by checking a group of individuals at the same time. Now we can handle much larger problems, which is the case of most of the real life problems. This fusion between CUDA and genetic algorithms helps pushing forward the boundaries of evolutionary learning by overcoming technical boundaries M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 23 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Further work Extend our methodology to use more than one GPGPU at the time. Develop models to determine the limits of the usage of CUDA. Disable the CUDA evaluation when the training set is small. Study the impact of the ILAS in the accuracy. Adapt the CUDA methodology to other evolutionary learning systems. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 24 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Bacardit, J. (2004). Pittsburgh Genetics-Based Machine Learning in the Data Mining era: Representations, generalization, and run-time. PhD thesis, Ramon Llull University, Barcelona, Spain. Bacardit, J., Burke, E., and Krasnogor, N. (2009). Improving the scalability of rule-based evolutionary learning. Memetic Computing, 1(1):55–67. Bacardit, J. and Llorà, X. (2009). Large scale data mining using genetics-based machine learning. In GECCO ’09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference, pages 3381–3412, New York, NY, USA. ACM. Langdon, W. B. and Harrison, A. P. (2008). GP on SPMD parallel graphics hardware for mega bioinformatics data mining. Soft Comput., 12(12):1169–1183. Loiacono, D. and Lanzi, P. L. (2009). Speeding up matching in XCS. In 12th International Workshop on Learning Classifier Systems. Maitre, O., Baumes, L. A., Lachiche, N., Corma, A., and Collet, P. (2009). Coarse grain parallelization of evolutionary algorithms on GPGPU cards with EASEA. In Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages 1403–1410, Montreal, Québec, Canada. ACM. Rissanen, J. (1978). Modeling by shortest data description. Automatica, vol. 14:465–471. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 24 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Sastry, K. (2005). Principled efficiency enhancement techniques. Genetic and Evolutionary Computation Conference - GECCO 2005- Tutorial. Stout, M., Bacardit, J., Hirst, J. D., and Krasnogor, N. (2008). Prediction of recursive convex hull class assignments for protein residues. Bioinformatics, 24(7):916–923. Venturini, G. (1993). SIA: a supervised inductive algorithm with genetic search for learning attributes based concepts. In Brazdil, P. B., editor, Machine Learning: ECML-93 - Proceedings of the European Conference on Machine Learning, pages 280–296. Springer-Verlag. M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 25 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Questions or comments? M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 25 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work Iterative Rule Learning IRL has been used for many years in the ML community, with the name of separate-and-conquer Algorithm 4.1: I TERATIVE RULE L EARNING(Examples) Theory ← ∅ whileExample = ∅ Rule ← FindBestRule(Examples)  Covered ← Cover (Rule, Examples)    if RuleStoppingCriterion(Rule, Theory , Examples)  do  then exit  Examples ← Examples − Covered    Theory ← Theory ∪ Rule  return (Theory ) M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 26 / 27
    • BioHEL GBML System BioHEL using CUDA Conclusions Experiments and results Further Work Conclusions and Further Work BioHEL fitness function Coverage term penalizes rules that do not cover a minimum percentage of examples system M. Franco, N. Krasnogor, J. Bacardit. Uni. Nottingham Speeding up Evolutionary Learning using GPGPUs 27 / 27