SlideShare a Scribd company logo
1 of 1
Download to read offline
• Green Infrastructures (GIs) are being used more today to
advance urban infrastructure systems performance, reduce
associated costs, and diminish greenhouse gasses emission
to achieve urban sustainability.
• GIs provide control at the area of generation, so there
won’t be capital and performance cost and emissions in
conveyance and storage facilities (examples in Fig 1).
Fig 1: Rainwater Harvesting Cistern as an example of GIs (left) and CSO storage tunnel as an
example of traditional urban water facilities (right)
• However, choosing an intellectually robust plan among the
millions of possible LID implementation plans according to
the variation in type, size and location through long-term
continuous simulation is a computational challenge.
Accelerating Stormwater Infrastructure Design
With High-throughput Computing Using HTCondor
Hassan Tavakol1, Steven Burian1, Maryam Imani2, Andrew Duncan2, Raziyeh Farmani2, David Butler2
1Urban Water Group, University of Utah; 2Centre for Water Systems, University of Exeter
RESULTSINTRODUCTION METHODS
is the combined sewer network of the
East Side Interceptor System at the City of Toledo
(4th most populous city in Ohio, US). This system
extends along the east side of the Maumee
River and serves a total of 25,000 acres, of
which 2,200 acres are combined. As a part
of LTCP (Long Term Control Plan), a model was
developed in the US-EPA SWMM (Storm
Water Management Model) 5.0 (Fig 3). 5years
continuous rainfall (1997-2001) was
Selected as the representative period.
was performed as the 1st scenario
to accelerate the optimization. In
this scenario, all the 4 physical
cores (Core i7 – 2600 CPU@3.44GHz)
were used to evaluate the objective
function through the parfor
function in MATLAB 2013b.
high-throughput computing framework - 8.0.5
CONCLUSION AND FUTURE WORKS
• Application of the presented approach in the studied area
could lead to the same CSO control with 2.2 Million $ less!!!
• More objective functions, more GIs, bigger systems?!
REFERENCE
Deb K, Pratap A, Agarwal S, (2002) A fast and elitist multiobjective genetic
algorithm NSGA-II[J]. Evolutionary Computation. 6(2): 182-197
Graham Saunders, Dr. Defne Apul, Andy Stepnick, Jay Devkota
ACKNOWLEDGEMENTS
STUDY AREA
• This research is guided by the question: “How Multi-
Objective Optimization of RainWater Harvesting (RWH)
storage to minimize the Combined Sewer Overflow (CSO)
volume and implementation cost can be accelerated in a
complex system and long-term continuous data?”
• In such system, evaluating the objective functions takes
more than 99% of the time of optimization. Therefore,
parallelization of CSO modeling for different individuals in
each generation of evolutionary algorithm (illustrated in Fig
2) was tested as an answer of the above question.
Fig 2: Parallelizing CSO modeling for different individuals during the evolutionary algorithm
generations
RESEARCH QUESTION
Fig 3: SWMM
model of the
studied
combined
sewer
network
Parallelization for
different individuals
in same generation
Fig 4: CPU cores usage in regular un-
parallelized optimization process (left)
and in parallelized case (right)
Fig 5: Schematic of the
HTCondor network
edition - was used as the 2nd scenario. It
provided 68 potential nodes through 4
researchers’ computers and 5 undergrad’s
computer site at university of Exeter (Fig 5)
when their CPU was idle for 15 min. The jobs
that were suspended (due to the owner use) were
resubmitted to different machines to avoid
holding the generation progress.
(Non-dominated Sorting Genetic
Objective
Functions
- Total CSO volume
- RWH implem. cost
Decision
Variables
41 (RWH storage in
each subcatchment)
Max Gen 500
Pop Size 40
framework was developed in
this research to connect NSGA-
II to SWMM, and automate the
job submission process (Fig 6).
was also
used based on one 200Lit rain
barrel per building for
comparison purposes.
for generation#i
GA operations %selection,…
for individual#j
create SWMM_input(i,j)
submit condor_job(i,j)
end
wait until all jobs are done
for individual#j
update cost objfun(i,j)
read SWMM_output(i,j)
update CSO objfun(i,j)
end
end
Table1: NSGA-II details
Fig 6: Developed framework
Algorithm) was used via NGPM-1.4
(NsGa-II Program in Matlab) developed
by Mathwork (details in Table1).
0
200
400
600
HTCondor CPU
Parallelization
Time(hr)
• HTCondor could accelerate the
optimization remarkably (Fig 7),
especially during weekend.
• Studying the convergence
process (Fig 8) showed that Pareto
front after 175th generation has
had minimal improvement.
0
1
2
3
4
5
6
7
8
9
10
1200 1250 1300 1350 1400
ImplementationCost(Million$)
CSO Volume (Million Galon)
Initial Generation
Generation#50
Generation#100
Generation#150
Generation#175
Generation#200
Fig 7: Comparing the elapsed time in
the two accelerating methods
Fig 8: Convergence process, Pareto front in
the last generation, and the selected
individual for comparison purposes (the one
with the grey filling colour)
Selecting the best individual among the Pareto front is not a
goal of this research. Stakeholders can choose it base on the
financial constraints, water quality requirements, and the
weights that they consider
for the two objectives. The
one with closest CSO to
Simple Design is selected
just for comparison (Fig 9).
0
1
2
3
4
0
500
1000
1500
2000
Million$
MillionGallon
CSO Volume
Cost
Fig 9: Comparing design approaches

More Related Content

What's hot

What's hot (14)

Design and Estimation of Rain Water Harvesting Scheme in VIVA Institute of T...
Design and Estimation of Rain Water Harvesting Scheme in VIVA  Institute of T...Design and Estimation of Rain Water Harvesting Scheme in VIVA  Institute of T...
Design and Estimation of Rain Water Harvesting Scheme in VIVA Institute of T...
 
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUDIMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
 
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUDIMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
IMPROVING REAL TIME TASK AND HARNESSING ENERGY USING CSBTS IN VIRTUALIZED CLOUD
 
The Development of an Application Conceived for the Design, Feasibility Study...
The Development of an Application Conceived for the Design, Feasibility Study...The Development of an Application Conceived for the Design, Feasibility Study...
The Development of an Application Conceived for the Design, Feasibility Study...
 
General instructions
General instructionsGeneral instructions
General instructions
 
Air 2014 energy_efficiency_forum
Air 2014 energy_efficiency_forumAir 2014 energy_efficiency_forum
Air 2014 energy_efficiency_forum
 
Tasseled Cap Transformation Techniques Reference Writing
Tasseled Cap Transformation Techniques Reference WritingTasseled Cap Transformation Techniques Reference Writing
Tasseled Cap Transformation Techniques Reference Writing
 
Tasseled Cap transformation Technique in ArcGIS
Tasseled Cap transformation Technique in ArcGISTasseled Cap transformation Technique in ArcGIS
Tasseled Cap transformation Technique in ArcGIS
 
Poster
PosterPoster
Poster
 
20120140507001
2012014050700120120140507001
20120140507001
 
Pitch deck
Pitch deckPitch deck
Pitch deck
 
Windfarm shell hackathon
Windfarm shell hackathonWindfarm shell hackathon
Windfarm shell hackathon
 
cloud schedualing
cloud schedualingcloud schedualing
cloud schedualing
 
09
0909
09
 

Similar to Hassan - Condor _48_x_36

Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...
Zbigniew Jerzak
 
Super-linear speedup for real-time condition monitoring using image processi...
Super-linear speedup for real-time condition monitoring using  image processi...Super-linear speedup for real-time condition monitoring using  image processi...
Super-linear speedup for real-time condition monitoring using image processi...
IJECEIAES
 
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
IJCSEA Journal
 
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
ijsrd.com
 
An optimized cost-based data allocation model for heterogeneous distributed ...
An optimized cost-based data allocation model for  heterogeneous distributed ...An optimized cost-based data allocation model for  heterogeneous distributed ...
An optimized cost-based data allocation model for heterogeneous distributed ...
IJECEIAES
 

Similar to Hassan - Condor _48_x_36 (20)

Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...
 
Distributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data AnalysisDistributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data Analysis
 
High Dimensionality Structures Selection for Efficient Economic Big data usin...
High Dimensionality Structures Selection for Efficient Economic Big data usin...High Dimensionality Structures Selection for Efficient Economic Big data usin...
High Dimensionality Structures Selection for Efficient Economic Big data usin...
 
Heterogeneous computing with graphical processing unit: improvised back-prop...
Heterogeneous computing with graphical processing unit:  improvised back-prop...Heterogeneous computing with graphical processing unit:  improvised back-prop...
Heterogeneous computing with graphical processing unit: improvised back-prop...
 
Design and Simulation of a Water Supply System for Eramala Panchayat
Design and Simulation of a Water Supply System for Eramala PanchayatDesign and Simulation of a Water Supply System for Eramala Panchayat
Design and Simulation of a Water Supply System for Eramala Panchayat
 
Super-linear speedup for real-time condition monitoring using image processi...
Super-linear speedup for real-time condition monitoring using  image processi...Super-linear speedup for real-time condition monitoring using  image processi...
Super-linear speedup for real-time condition monitoring using image processi...
 
An enhanced adaptive scoring job scheduling algorithm with replication strate...
An enhanced adaptive scoring job scheduling algorithm with replication strate...An enhanced adaptive scoring job scheduling algorithm with replication strate...
An enhanced adaptive scoring job scheduling algorithm with replication strate...
 
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
 
Effective Utilization of CPU
Effective Utilization of CPUEffective Utilization of CPU
Effective Utilization of CPU
 
D0212326
D0212326D0212326
D0212326
 
Sustainable Development using Green Programming
Sustainable Development using Green ProgrammingSustainable Development using Green Programming
Sustainable Development using Green Programming
 
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
ENERGY EFFICIENT SCHEDULING FOR REAL-TIME EMBEDDED SYSTEMS WITH PRECEDENCE AN...
 
Design and testing of systolic array multiplier using fault injecting schemes
Design and testing of systolic array multiplier using fault injecting schemesDesign and testing of systolic array multiplier using fault injecting schemes
Design and testing of systolic array multiplier using fault injecting schemes
 
Parallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix MultiplicationParallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix Multiplication
 
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
 
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
Artificial Neural Network Based Graphical User Interface for Estimation of Fa...
 
A Novel Approach in Scheduling Of the Real- Time Tasks In Heterogeneous Multi...
A Novel Approach in Scheduling Of the Real- Time Tasks In Heterogeneous Multi...A Novel Approach in Scheduling Of the Real- Time Tasks In Heterogeneous Multi...
A Novel Approach in Scheduling Of the Real- Time Tasks In Heterogeneous Multi...
 
GRID COMPUTING
GRID COMPUTINGGRID COMPUTING
GRID COMPUTING
 
IRJET- Enhance Dynamic Heterogeneous Shortest Job first (DHSJF): A Task Schedu...
IRJET- Enhance Dynamic Heterogeneous Shortest Job first (DHSJF): A Task Schedu...IRJET- Enhance Dynamic Heterogeneous Shortest Job first (DHSJF): A Task Schedu...
IRJET- Enhance Dynamic Heterogeneous Shortest Job first (DHSJF): A Task Schedu...
 
An optimized cost-based data allocation model for heterogeneous distributed ...
An optimized cost-based data allocation model for  heterogeneous distributed ...An optimized cost-based data allocation model for  heterogeneous distributed ...
An optimized cost-based data allocation model for heterogeneous distributed ...
 

Hassan - Condor _48_x_36

  • 1. • Green Infrastructures (GIs) are being used more today to advance urban infrastructure systems performance, reduce associated costs, and diminish greenhouse gasses emission to achieve urban sustainability. • GIs provide control at the area of generation, so there won’t be capital and performance cost and emissions in conveyance and storage facilities (examples in Fig 1). Fig 1: Rainwater Harvesting Cistern as an example of GIs (left) and CSO storage tunnel as an example of traditional urban water facilities (right) • However, choosing an intellectually robust plan among the millions of possible LID implementation plans according to the variation in type, size and location through long-term continuous simulation is a computational challenge. Accelerating Stormwater Infrastructure Design With High-throughput Computing Using HTCondor Hassan Tavakol1, Steven Burian1, Maryam Imani2, Andrew Duncan2, Raziyeh Farmani2, David Butler2 1Urban Water Group, University of Utah; 2Centre for Water Systems, University of Exeter RESULTSINTRODUCTION METHODS is the combined sewer network of the East Side Interceptor System at the City of Toledo (4th most populous city in Ohio, US). This system extends along the east side of the Maumee River and serves a total of 25,000 acres, of which 2,200 acres are combined. As a part of LTCP (Long Term Control Plan), a model was developed in the US-EPA SWMM (Storm Water Management Model) 5.0 (Fig 3). 5years continuous rainfall (1997-2001) was Selected as the representative period. was performed as the 1st scenario to accelerate the optimization. In this scenario, all the 4 physical cores (Core i7 – 2600 CPU@3.44GHz) were used to evaluate the objective function through the parfor function in MATLAB 2013b. high-throughput computing framework - 8.0.5 CONCLUSION AND FUTURE WORKS • Application of the presented approach in the studied area could lead to the same CSO control with 2.2 Million $ less!!! • More objective functions, more GIs, bigger systems?! REFERENCE Deb K, Pratap A, Agarwal S, (2002) A fast and elitist multiobjective genetic algorithm NSGA-II[J]. Evolutionary Computation. 6(2): 182-197 Graham Saunders, Dr. Defne Apul, Andy Stepnick, Jay Devkota ACKNOWLEDGEMENTS STUDY AREA • This research is guided by the question: “How Multi- Objective Optimization of RainWater Harvesting (RWH) storage to minimize the Combined Sewer Overflow (CSO) volume and implementation cost can be accelerated in a complex system and long-term continuous data?” • In such system, evaluating the objective functions takes more than 99% of the time of optimization. Therefore, parallelization of CSO modeling for different individuals in each generation of evolutionary algorithm (illustrated in Fig 2) was tested as an answer of the above question. Fig 2: Parallelizing CSO modeling for different individuals during the evolutionary algorithm generations RESEARCH QUESTION Fig 3: SWMM model of the studied combined sewer network Parallelization for different individuals in same generation Fig 4: CPU cores usage in regular un- parallelized optimization process (left) and in parallelized case (right) Fig 5: Schematic of the HTCondor network edition - was used as the 2nd scenario. It provided 68 potential nodes through 4 researchers’ computers and 5 undergrad’s computer site at university of Exeter (Fig 5) when their CPU was idle for 15 min. The jobs that were suspended (due to the owner use) were resubmitted to different machines to avoid holding the generation progress. (Non-dominated Sorting Genetic Objective Functions - Total CSO volume - RWH implem. cost Decision Variables 41 (RWH storage in each subcatchment) Max Gen 500 Pop Size 40 framework was developed in this research to connect NSGA- II to SWMM, and automate the job submission process (Fig 6). was also used based on one 200Lit rain barrel per building for comparison purposes. for generation#i GA operations %selection,… for individual#j create SWMM_input(i,j) submit condor_job(i,j) end wait until all jobs are done for individual#j update cost objfun(i,j) read SWMM_output(i,j) update CSO objfun(i,j) end end Table1: NSGA-II details Fig 6: Developed framework Algorithm) was used via NGPM-1.4 (NsGa-II Program in Matlab) developed by Mathwork (details in Table1). 0 200 400 600 HTCondor CPU Parallelization Time(hr) • HTCondor could accelerate the optimization remarkably (Fig 7), especially during weekend. • Studying the convergence process (Fig 8) showed that Pareto front after 175th generation has had minimal improvement. 0 1 2 3 4 5 6 7 8 9 10 1200 1250 1300 1350 1400 ImplementationCost(Million$) CSO Volume (Million Galon) Initial Generation Generation#50 Generation#100 Generation#150 Generation#175 Generation#200 Fig 7: Comparing the elapsed time in the two accelerating methods Fig 8: Convergence process, Pareto front in the last generation, and the selected individual for comparison purposes (the one with the grey filling colour) Selecting the best individual among the Pareto front is not a goal of this research. Stakeholders can choose it base on the financial constraints, water quality requirements, and the weights that they consider for the two objectives. The one with closest CSO to Simple Design is selected just for comparison (Fig 9). 0 1 2 3 4 0 500 1000 1500 2000 Million$ MillionGallon CSO Volume Cost Fig 9: Comparing design approaches