SlideShare a Scribd company logo
1 of 8
Download to read offline
International Journal of Electrical and Computer Engineering (IJECE)
Vol. 10, No. 5, October 2020, pp. 4910~4917
ISSN: 2088-8708, DOI: 10.11591/ijece.v10i5.pp4910-4917 ļ² 4910
Journal homepage: http://ijece.iaescore.com/index.php/IJECE
Earlier stage for straggler detection and handling using
combined CPU test and LATE methodology
Anwar H. Katrawi1
, Rosni Abdullah2
, Mohammed Anbar3
, Ammar Kamal Abasi4
1,3
National Advanced IPv6 Center (Nav6), Universiti Sains Malaysia, Malaysia
2,4
School of Computer Sciences, Universiti Sains Malaysia, Malaysia
Article Info ABSTRACT
Article history:
Received Oct 15, 2019
Revised Mar 17, 2020
Accepted Mar 30, 2020
Using MapReduce in Hadoop helps in lowering the execution time and
power consumption for large scale data. However, there can be a delay in job
processing in circumstances where tasks are assigned to bad or congested
machines called "straggler tasks"; which increases the time, power
consumptions and therefore increasing the costs and leading to a poor
performance of computing systems. This research proposes a hybrid
MapReduce framework referred to as the combinatory late-machine (CLM)
framework. Implementation of this framework will facilitate early and timely
detection and identification of stragglers thereby facilitating prompt
appropriate and effective actions.
Keywords:
Big data
Combinatory late-machine
Hadoop
Map reduce
Straggler Copyright Ā© 2020 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Anwar H. Katrawi,
National Advanced IPv6 Center (Nav6),
Universiti Sains Malaysia,
11800 USM, Penang, Malaysia.
Email: akatrawi@student.usm.my
1. INTRODUCTION
A significant amount of data (big data) is stored and transferred online by thousands of companies,
organizations, and individuals. This largely unstructured data is difficult to analyze and process using
conventional database management tools which creates new challenges in the analysis and the storage of
data [1]. Recently, there has been an increasing interest in key areas such as real-time data extraction,
which reveals an urgent need for bulk and strict performance constraints. Consequently, the adaptation of
huge data to implementations on distributed computing platforms is necessary. One way of doing this is to
adopt and implement the popular programming model known as MapReduce [2]. Its success lies in
simplicity, scalability, efficiency and extensibility that pushes the IT industry leaders such as Google, Yahoo,
Facebook and Amazon to extensively adopt MapReduce as a powerful and reliable tool for Big Data
processing. There are four factors, including processing, storing, visualization, and analyzing large data in
modern organizations and enterprises. MapReduce can run the applications on a parallel cluster of hardware
automatically. In addition; it can process terabytes and petabytes of data more rapidly [3,4]. Recently, it has
gained popularity in a wide range of applications due to its ability to provide a highly effective and efficient
framework for the parallel execution of the applications, data allocation in distributed database systems, and
fault tolerance network communications [5]. For instance, Google runs more than 10,000 distinct programs
using MapReduce including graph processing[6], text processing, machine learning, and statistical machine
translation. Moreover, the famous open-source Hadoop software framework for distributed storage and
processing of big data Figure 1 sets uses Mapreduce as a central tool to split the data, process it and make it
not only manageable but also available for usersā€™ consumption or further processing. Also, the Hadoop
MapReduce environment provides fault-tolerant solutions in case of hardware failures or software errors
during the execution of tasks [7].
Int J Elec & Comp Eng ISSN: 2088-8708 ļ²
Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi)
4911
Figure 1. Hadoop framework
According to the work presented by [8], Hadoop MapReduce has the ability to tolerate several types
of faults and they are as follows:
a. Nodes failure: A node in a MapReduce cluster may fail at any time. In this case, the JobTracker removes
this slave node from the list of nodes available and re-executes the tasks on other nodes. It can be
concluded that a node is declared failing if at least one task launched on it has failed. At the time,
the JobTracker checks if the node in question should not be blacklisted. If a slave node is "blacklisted",
the JobTracker will no longer assign it map or reduce. It can be removed from this list if his behavior
becomes normal and does not commit faults during a certain time interval.
b. Software Failure: A task may stop because of an error or exception in the mapping or reduction program.
In this case, the JobTracker orders the re-execution of the failed task to a limited number of attempts
(four by default), beyond which the task and the job of the task is considered faulty.
c. Stopped Task: As an example, the process of running a task may stop unexpectedly due to a transient bug
in the underlying virtual machine. In this case, the JobTracker show in Figure 2 will be notified and it will
reorder the job as described above.
d. Blocked Task: Itā€™s considered faulty if after some timea mapping or reduction task remains blocked
without any progress; in this case, the JobTracker orders to kill the process running this task.
e. Delayed Tasks:When some tasks are unexpectedly taking longer execution time compared with
the average execution time, these tasks called stragglers.
Figure 2. Job tracker
Several studies have been devoted to improving the tolerance of systems to faults. For instance,
Microsoft reveals that when the CPU and core partsā€™ errors occur in one million customersā€™ computers,
Hadoop does not have the capacity to deal with these types of potential errors. This also includes other types
of errors related to the task itself (when one or more of these tasks take longer or stops before the work
outcomes are realized).
Problem statement. The delayed tasks are called stragglers and play a key role in increasing
the execution time of big data and energy consumption. Following the MapReduce framework, stragglers
refer to the tasks that take a longer time to be executed compared to other tasks. There are various techniques
of detecting and handling stragglers such as Dolly, the Hadoop native scheduler, MonTool, LATE and
Mantri. Regardless of the techniques already in place, straggler detection remains problematic in the field of
data analytics.
ļ² ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917
4912
The proposed solution. We propose an algorithm that facilitates the calculation of straggler tolerance
threshold using CPU test and LATE methodology. Our approach considers crucial issues such as the QoS
with a major focus on the timing constraints, the progress of task execution, and the usage of cluster
resources. Even though task selection does not appear to be a big problem in the initial stages, we have been
able to show that it is a big issue that requires close attention. Therefore, we recommend identifying those
tasks that lead to the longest response times. We also recommend that this needs to be done as early as
possible so that there could be no later surprises. Taking these into account, it is possible to see that our
LATE methodology is based on the estimate of the time that is left with the goal of early detection of
the tasks that are running slowly. In summary, the methodology is based on making decisions early, using
the finishing times and not the progress rates, not assigning speculative tasks to slow nodes and optimization
of resource utilization.
Significance of the research. In the current study, a method is proposed with the purpose of
addressing the straggler problem. We propose an algorithm which can enable calculation of threshold of
straggler tolerance earlier using CPU test and Longest Approximate Time to End (LATE) methodology.
Our approach focuses on the timing constraints, progress of task execution, and the usage of cluster
resources. In this work, our major contribution is that unlike other studies that assume that it is hard to have
a correlation between task execution and node status, we show that it is possible and feasible to have
a correlation and detect stragglers using LATE methodology and CPU test. Our methodology is simple
enough and easy to accommodate with low overheads.
Organization of the research. The rest of this paper is organized as follows: Section 2 covers
the literature review, section 3 covers the methodology, section 4 the results and discussion, section 5 covers
the proposed solution/recommendation, and section 6 covers the conclusion.
2. LITERATURE REVIEW
Hadoop and Map Reduce are among the most commonly used frameworks when it comes to task
execution across several nodes for optimal performance. Even though the frameworks have become popular,
they still face several challenges when it comes to the effectiveness of tasks execution [8]. Specifically,
achieving predictable execution has become problematic because of stragglers. Due to stragglers, tasks
execution takes longer to complete than originally anticipated [9]. Such delays are undesirable because they
result in reduced service performance and can also potentially violate QoS (Quality of Service) requirements
concerning time taken to complete tasks. For service providers, tasks that take more time to complete lead to
reduced availability of systems and cause jobs to consume more time. Stragglers have become common
especially in cloud data centers [10].
Therefore, it is vital to detect and mitigate them promptly. Additionally, centers with large
computing infrastructure can also experience delays that can lead to ineffective job execution. Also, large
data centers have a high intake of service creation which make them vulnerable to stragglers. There are
several root causes to stragglers including resource contention, hardware heterogeneity, background network
traffic, and operating system related-level causes [11]. Considerable effort has been made to study stragglers.
Over the years, the size of computing infrastructure and jobs executed have continued to grow which has
dramatically increased the impact of stragglers. Stragglers are known to extend job execution substantially
which negatively affects the ā€œConsumer Service Level Agreementā€ and QoS performance requirements.
In a study conducted by Bortnikov, Frank, Hillel, and Rao, the authors propose two ways of dealing with
stragglers namely tolerance and avoidance [12]. However, avoiding stragglers is difficult since it is
impractical to pursue. Therefore, straggler tolerance is the approach adopted by most stakeholders.
In straggler tolerance, the execution progress of a task is monitored using a percentage score made up of
values ranging from 0 to 1 which represent start and completion. Currently, the approaches used for straggler
detection can either be described as offline or online analytics [13].
Nonetheless, it is worth noting that online detection can occur too late during the execution cycle of
a task. Therefore, stragglers cannot be prevented from running slower even after the implementation of
speculative copies. On the other hand, offline approaches are normally applied to avoid stragglers.
This approach is seen as less feasible and thus it is uncommon. However, better results can be achieved by
combining both online and offline approaches. When used together, they can significantly help to improve
the effectiveness of ā€œstraggler detectionā€.
2.1. Related works
For Straggler Detection, many techniques have been developed. Ouyang et al. proposes an algorithm
based on the progress score of task execution that enables dynamic threshold detection for straggler
tasks [14]. This strategy has improved performance significantly by reducing job execution by 44 percent.
Int J Elec & Comp Eng ISSN: 2088-8708 ļ²
Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi)
4913
In a study conducted by Zaharia et al., the authors propose a new Straggler Detection that takes into account
both the progress score and elapsed time with the objective of improving the progress score strategy [15].
The authors determine that the strategy does not have the capacity of determining how fast a task runs among
different tasks starting at different times hence the need for improvement. Dean and Ghemawat have adopted
a technique called Speculative Execution in which they launch copies of the straggler on alternative nodes
with the aim of improving performance [16]. Google acknowledges that Speculative Execution improves job
execution by 44 percent. However, this technique can reduce the overall throughput due to the duplication of
tasks. Therefore, some Hadoop administrators prefer not to use the Speculative Execution option [17, 18].
Yanfei et al proposed another technique for Straggler handling which consists of ending the delayed tasks
and reassigning them to another node without stragglers, however, Guo, Rao, Jiang, and Zhou disagrees with
this technique and argue that it results in wastage of resources and therefore increases energy consumption [19].
In a similar study, Zhou, Li, Yang, Jia, and Li propose a technique known as ā€œBigRootsā€ which
involves incorporating both system features and framework for the analysis of root causes of stragglers
especially in big data systems [20]. The authors establish that ā€œBigRootsā€ is effective when it comes to
identifying the ā€œroot causesā€ of stragglers which can significantly help in optimizing performance.
An examination by Phan attempted to come up with ā€œenergy efficient straggler mitigationā€ technique for big
data applications especially in the cloud environment [21-23]. The framework employed by the author takes
into account how heterogeneity of resources affect the performance and energy consumption of big
data applications.
In another studies by Harlap et al., Kim, W., the authors sought to solve the straggler problem for
parallel ML [24, 25]. The authors combined a more ā€œflexible synchronization modelā€ together with
the experiments involving real straggler behaviors and synthetic straggler behaviors to come up with near-ideal
run times across all the straggler patterns they tested. Similarly, Yadwadkar et al. came up with a framework
called ā€œWranglerā€ which could predict when stragglers were going to occur and aid in making scheduling
decisions [26]. The formulations employed by the authors captured the shared structure in their data so that it
could improve the generalization performance of their data.
3. METHODOLOGY
As noted earlier, we estimate the time remaining for each task based on the process score derived
from Hadoop. In practice, this heuristic works well. However, we want to point out that there are incidences
when it can backfire. When this happens, the heuristic can provide incorrect estimates and give results that
a task launched later finishes earlier. To demonstrate the delay, we assume that the progress of a task grows
by five percent during the first phase. We assume that during this first phase, the total score is fifty percent
and that the rate reduces by one percent in the second phase. In the first phase, it is expected that the task will
take ten seconds and fifty seconds in the second phase to produce a total of sixty seconds. When two copies
of the same task are launched at the same time, the first task is denoted by T1, the second is noted by T2 and
the first task starts at time 0 while the next starts after ten seconds. The progress rate is checked after twenty
seconds whereby twenty seconds, it is expected that T1 will have finished the first phase and will be through
a fifth of the second phase. Therefore, it will have a progress score of sixty percent. Its rate of progress will
be 60%/20s=3%/s. On the other hand, T2 will just be through with the first and its score will be 50%. Its rate
will, therefore, be 50%/10s=5%/s. The estimated time remaining for T1 will be (100%-60%)/(3%/s)=13.3s.
For T2, the estimated time left will be (100%-50%)/(5%/s)=10s. Therefore, the heuristic will illustrate that
T1 will take a longer time to run compared to T2. However, in reality, T2 will finish second compared to
T1. We also determined the criteria that we could use to identify stragglers. We took the finish time for every
task to be represented by (1).
EFš‘‡
š‘–
š‘—
=tk +
1āˆ’PS(task
š‘–
š‘—
)
PS(task
š‘–
š‘—
)
(tk-t0) (1)
In this instance, EFš‘‡
š‘–
š‘—
represents the estimated finish time. PS is used to represent the progress score
for a š‘”ask
š‘–
š‘—
whereas t0 is starting time while tk is the timestamp recording for PS(task
š‘–
š‘—
). Our proposed
LATE methodology takes into account the scope of data and the speed of processing data. By taking these
into account, we can determine the pattern of straggler detection and correlate this with the attributes of
a system normally hypothesized to give them the ability to cause stragglers. Some of the attributes include
resource utilization (memory, CPU, disk), hardware faults, unhandled requests, among others. When we
ļ² ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917
4914
combine these runtime features with dynamic information, we can come up with better prediction and
learning models that can discriminate stragglers promptly and lower the number of failures occurring due to
late timing.
4. RESULTS AND DISCUSSION
This research has identified the need for improving the effectiveness of MapReduce processes to
facilitate costs reduction and maximum utilization of resources. Performance improvement can be achieved
by eliminating inefficiencies brought about by the existence of stragglers. Stragglers have the effect of
resulting in poor user code and uneven aggregation of workloads. Poor user code is the product of looping
conditions that are designed ineffectively and uneven aggregation of workloads results from extreme
co-allocation of workloads due to inefficient scheduling. In the case of MapReduce tasks with a massive
number of write and read queries, it is common for file requests to be overloaded thereby leading to
inefficiency in handling the requests. It is notable that once the threshold of the master node is surpassed,
stragglers set in which implies that the waiting queue for requests becomes long. If the requests continue
increasing, then the master node becomes overloaded thereby further slowing down the handling process of
the requests. This experiment focused on investigating how the occurrence of stragglers is affected by
contention of resources. An observation of the occurrence of stragglers that took place over a period of
20 days yielded the results presented in Figure 3.
Figure 3. Graphical presentation of the findings
In the analysis, millions of tasks processed in five hundred servers in Cloud Datacenter [27] were
investigated. The inclusion criteria involved:
a. Servers whose tasks had DoS-Index values greater than or equal to 10
b. Servers whose utilization of CPU was greater than or equal to 80%
c. Latency from file processing greater than 400ms which translates to slow handling of write and read
requests
From the investigation, it came to our attention that 42% of stragglers are brought about by
overloading of disks while 59% of stragglers exist under high server CPU conditions. The findings also
revealed that slow handling of requests was responsible for the occurrence of 34.3% of stragglers.
From the findings, it is evident that the existence of stragglers is significantly caused by high utilization of
resources. It was also observed that stragglers can be caused by other factors such as the conditions of
the network.
5. RECOMMENDATION/PROPOSED SOLUTION
In this paper, we propose a combined strategy called COMBINATORY LATE-MACHINE that tests
the CPU machine. The CPU machine is tested to determine its vulnerability to stragglers. To achieve this,
the CPU and the RAM are tested to determine machines that have a CPU usage less than 85. Those found to
meet this condition are dropped and the ones that exceed 85 are selected and placed in a performance chart
Int J Elec & Comp Eng ISSN: 2088-8708 ļ²
Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi)
4915
starting with those that have the highest performance. If for any reason the CPU/Memory is lower than
a certain threshold, the jobs will automatically be redirected to another machine with a higher CPU/Memory
performance. This stage is run once and helps in creating a list of performances for all the machines.
After this step, Straggler Detection and handling are initiated at the same time using LATE algorithm.
The algorithm has several benefits that make it suitable for heterogeneous jobs since it re-executes only
the slowest tasks. The primary advantage of the proposed methodology is the increasing probability to detect
the straggler machines and straggler tasks with the same algorithm in earlier stages. Our LATE scheduler is
designed such that it includes all the features needed for it to function well in a realistic environment.
The major insight behind the LATE algorithm is that tasks believed to finish last can be executed at any time
in the future because this is the best way through which the response time can be improved. The framework
of the proposed combinatory Late-Machine strategy is illustrated in Figure 4.
Figure 4. Proposed combinatory late-machine framework
5.1. Limitations and future research
Our observation did not focus on the conditions of the memory capacity as well as overlapping of
conditions. Therefore, this warrants the need for further investigations.
6. CONCLUSION
In this paper, we proposed the Combinatory Late-Machine (CLM) strategy to identify (in earlier
stages) the straggler for both nodes and tasks, and the optimal node for re-execution of slow tasks.
The overall execution time is improved significantly using this technique compared to traditional job
schedulers. In the future, the CLM strategy will be tested with Hadoop to evaluate the efficiency of this
technique. The findings provide new insights into early straggler detection. However, the current strategy has
its shortcomings and there is a need for exhaustive analytics to establish the relationship between stragglers
and the contention of resources. The proposed algorithm could be used for research as well as in the industry
to improve the time and cost for big data processing.
ACKNOWLEDGMENT
The authors would like to thank all the research participants for their time, effort, and contribution to
the research
ļ² ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917
4916
REFERENCES
[1] E. A. Mohammed, et al., ā€œApplications of the MapReduce programming framework to clinical big data analysis:
current landscape and future trends,ā€ Bio Data Mining, vol. 7, no. 1, pp. 22-44, 2014.
[2] S. ValvĆ„g, et al., ā€œCogset: a high performance MapReduce engine,ā€ Concurrency and Computation: Practice and
Experience, vol. 25, no. 1, pp. 2-23, 2012.
[3] Sumalatha, S., and Subramanyam, R. B. V. ā€œDistributed mining of high utility time interval sequential patterns
using mapreduce approach,ā€ Expert Systems with Applications, vol. 141, 112967, Mar. 2020.
[4] Meddah, I. H., and Belkadi, K. ā€œParallel Distributed Patterns Mining Using Hadoop MapReduce Framework,ā€
International Journal of Grid and High Performance Computing, vol. 9, no. 2, pp. 70-85, 2017.
[5] S. Khezr and N. J. Navimipour, ā€œMapReduce and Its Applications, Challenges, and Architecture: a Comprehensive
Review and Directions for Future Research,ā€ Journal of Grid Computing, vol. 15, no. 3, pp. 295-321, 2017.
[6] Prameela De'vi. Chillakuru, T. Kumanan, CH. Sarada Devi, ā€œContent based Retrieval Management Systems in Web
Engineering,ā€ International Journal of Recent Technology and Engineering (IJRTE), vol. 8, no. 2S11, pp. 81-93,
Sep. 2019
[7] A. Y. Pigul, ā€œComparative Study Parallel Join Algorithms for MapReduce environment,ā€ Proceedings of the
Institute for System Programming of RAS, vol. 23, pp. 285-306, 2012.
[8] I. Hashem, et al., ā€œMapReduce scheduling algorithms: a review,ā€ The Journal of Supercomputing, 2018.
[9] K. Mitsuzuka, et al., ā€œProxy Responses by FPGA-Based Switch for MapReduce Stragglers,ā€ IEICE Transactions
on Information and Systems, vol. 101, no. 9, pp. 2258-2268, 2018.
[10] J. Rogoff, ā€œStragglers,ā€ Sewanee Review, vol. 124, no. 3, pp. 397-397, 2016.
[11] M. F. Aktas, et al., ā€œStraggler Mitigation by Delayed Relaunch of Tasks,ā€ ACM SIGMETRICS Performance
Evaluation Review, vol. 45, no. 2, pp. 248-248, 2018.
[12] E. Bortnikov, et al., ā€œPredicting execution bottlenecks in map-reduce clusters,ā€ Proceedings of the 4th USENIX
conference on Hot Topics in Cloud Computing, pp. 1-18, 2012.
[13] A. K. Abasi, et al., ā€œLink-based multi-verse optimizer for text documents clustering,ā€ Applied Soft Computing,
vol. 87, 2019.
[14] X. Ouyang, et al., ā€œStraggler Detection in Parallel Computing Systems through Dynamic Threshold
Calculation,ā€ 2016 IEEE 30th International Conference on Advanced Information Networking and Applications
(AINA), pp. 414-421, 2016.
[15] J. Xie, et al., ā€œImproving MapReduce performance through data placement in heterogeneous Hadoop
clusters,ā€ 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum
(IPDPSW), pp. 1-9, 2010.
[16] J. Dean and S. Ghemawat, ā€œMap Reduce: simplified data processing on large clusters,ā€ Communications of
the ACM, vol. 51, no. 1, p. 107, 2008.
[17] H. Wu, et al., ā€œA Heuristic Speculative Execution Strategy in Heterogeneous Distributed Environments,ā€ 2014
Sixth International Symposium on Parallel Architectures, Algorithms and Programming, pp. 268-273, 2014.
[18] A. K. Abasi, et al., ā€œA Text Feature Selection Technique based on Binary Multi-Verse Optimizer for Text
Clustering,ā€ 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information
Technology (JEEIT), Amman, Jordan, pp. 1-6, 2019.
[19] Y. Guo, et al., ā€œFlexSlot: Moving Hadoop Into the Cloud with Flexible Slot Management,ā€ SC14: International
Conference for High Performance Computing, Networking, Storage and Analysis, 2014.
[20] H. Zhou, et al., ā€œBigRoots: An Effective Approach for Root-Cause Analysis of Stragglers in Big Data
System,ā€ IEEE Access, vol. 6, pp. 41966-41977, 2018.
[21] T. Phan, et al., ā€œA New Framework for Evaluating Straggler Detection Mechanisms in MapReduce,ā€ ACM
Transactions on Modeling and Performance Evaluation of Computing Systems, vol. 4, no. 3, pp. 1-23, 2019.
[22] Chaowei Yang, Qunying Huang, Zhenlong Li, Kai Liu & Fei Hu, ā€œBig Data and cloud computing: innovation
opportunities and challenges,ā€ International Journal of Digital Earth, vol. 10, no. 1, pp. 13-53, 2017. DOI:
10.1080/17538947.2016.1239771
[23] L. Greeshma, Pradeepini Gera, ā€œBig Data Analytics with Apache Hadoop MapReduce Framework,ā€ Indian Journal
of Science and Technology, Vol 9, no. 26, July 2016.
[24] A. Harlap, et al., ā€œAddressing the straggler problem for iterative convergent parallel ML,ā€ Proceedings of
the Seventh ACM Symposium on Cloud Computing - SoCC '16, pp. 98-111, 2016.
[25] Kim, W., Kim, Y., and Shim, K., ā€œParallel computation of k-nearest neighbor joins using MapRe-duce,ā€
In Proceedings of the IEEE International Conference on Big Data, pp. 696-705, 2016.
[26] N. Yadwadkar, et al., ā€œMulti-Task Learning for Straggler Avoiding Predictive Job Scheduling,ā€ Journal of Machine
Learning Research, vol. 17, pp. 1-37, 2016. [Online]. Available: http://jmlr.org/papers/volume17/15-149/15-149.pdf.
[27] Gigas the cloud computing company. [Online]. Available: https://gigas.com/en/cloud-datacenter.
Int J Elec & Comp Eng ISSN: 2088-8708 ļ²
Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi)
4917
BIOGRAPHIES OF AUTHORS
Anwar H. Katrawi received B. Sc. In computer science from al Mustansiriya university,
Iraq and M.Sc. in the computer information system from Arab Academy for Management,
Banking and Financial Sciences of Jordan. He is currently a PhD Candidate in the school of
Computer Sciences (NAV6) at Universiti Sains Malaysia. His research interests include Big
data, data warehouse, machine Learning and data analytics.
Rosni Abdullah is a professor in parallel computing and one of the national pioneers in the said
domain. She was appointed Dean of the School of Computer Sciences at Universiti Sains
Malaysia (USM) in June 2004, after having served as its Deputy Dean (Research) since 1999.
She is also the Head of the Parallel and Distributed Processing Research Group at the School
since its inception in 1994.
Mohammed F.R. Anbar received his bachelor of Computer System Engineering from
Al-Azhar University, Palestine and M.Sc. in Information Technology from Universiti Utara
Malaysia, Malaysia (UUM). He obtained his PhD.ā€Æin Advanced Internet Security and
Monitoringā€Æfrom University Sains Malaysia (USM). He is currently a senior lecturer at National
Advanced IPv6 Centre (NAv6), Universiti Sains Malaysia.
Ammar Kamal Abasi received B. Sc. in computer information system from Jordan university
of science and technology, and M.Sc. in the international business from the university of Jordan.
He is currently a PhD Candidate in the school of Computer Sciences at Universiti Sains
Malaysia. His research interests include evolutionary algorithms, nature-inspired computation,
and their applications to optimization problems.

More Related Content

What's hot

Graphlab Ted Dunning Clustering
Graphlab Ted Dunning  ClusteringGraphlab Ted Dunning  Clustering
Graphlab Ted Dunning ClusteringMapR Technologies
Ā 
Ieeepro techno solutions 2014 ieee java project - deadline based resource p...
Ieeepro techno solutions   2014 ieee java project - deadline based resource p...Ieeepro techno solutions   2014 ieee java project - deadline based resource p...
Ieeepro techno solutions 2014 ieee java project - deadline based resource p...hemanthbbc
Ā 
An effective classification approach for big data with parallel generalized H...
An effective classification approach for big data with parallel generalized H...An effective classification approach for big data with parallel generalized H...
An effective classification approach for big data with parallel generalized H...riyaniaes
Ā 
Improving the Performance of Mapping based on Availability- Alert Algorithm U...
Improving the Performance of Mapping based on Availability- Alert Algorithm U...Improving the Performance of Mapping based on Availability- Alert Algorithm U...
Improving the Performance of Mapping based on Availability- Alert Algorithm U...AM Publications
Ā 
Task Scheduling methodology in cloud computing
Task Scheduling methodology in cloud computing Task Scheduling methodology in cloud computing
Task Scheduling methodology in cloud computing Qutub-ud- Din
Ā 
Differentiating Algorithms of Cloud Task Scheduling Based on various Parameters
Differentiating Algorithms of Cloud Task Scheduling Based on various ParametersDifferentiating Algorithms of Cloud Task Scheduling Based on various Parameters
Differentiating Algorithms of Cloud Task Scheduling Based on various Parametersiosrjce
Ā 
A hybrid approach for scheduling applications in cloud computing environment
A hybrid approach for scheduling applications in cloud computing environment A hybrid approach for scheduling applications in cloud computing environment
A hybrid approach for scheduling applications in cloud computing environment IJECEIAES
Ā 
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...IRJET Journal
Ā 
REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...
 REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION... REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...
REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...ijiert bestjournal
Ā 
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENT
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENTA HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENT
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENTieijjournal
Ā 
Scalable scheduling of updates in streaming data warehouses
Scalable scheduling of updates in streaming data warehousesScalable scheduling of updates in streaming data warehouses
Scalable scheduling of updates in streaming data warehousesIRJET Journal
Ā 
Berlin Hadoop Get Together Apache Drill
Berlin Hadoop Get Together Apache Drill Berlin Hadoop Get Together Apache Drill
Berlin Hadoop Get Together Apache Drill MapR Technologies
Ā 
Role of Operational System Design in Data Warehouse Implementation: Identifyi...
Role of Operational System Design in Data Warehouse Implementation: Identifyi...Role of Operational System Design in Data Warehouse Implementation: Identifyi...
Role of Operational System Design in Data Warehouse Implementation: Identifyi...iosrjce
Ā 
Resisting skew accumulation
Resisting skew accumulationResisting skew accumulation
Resisting skew accumulationMd. Hasibur Rashid
Ā 
Fault tolerance on cloud computing
Fault tolerance on cloud computingFault tolerance on cloud computing
Fault tolerance on cloud computingwww.pixelsolutionbd.com
Ā 
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...Editor IJCATR
Ā 
A Baye's Theorem Based Node Selection for Load Balancing in Cloud Environment
A Baye's Theorem Based Node Selection for Load Balancing in Cloud EnvironmentA Baye's Theorem Based Node Selection for Load Balancing in Cloud Environment
A Baye's Theorem Based Node Selection for Load Balancing in Cloud Environmentneirew J
Ā 

What's hot (17)

Graphlab Ted Dunning Clustering
Graphlab Ted Dunning  ClusteringGraphlab Ted Dunning  Clustering
Graphlab Ted Dunning Clustering
Ā 
Ieeepro techno solutions 2014 ieee java project - deadline based resource p...
Ieeepro techno solutions   2014 ieee java project - deadline based resource p...Ieeepro techno solutions   2014 ieee java project - deadline based resource p...
Ieeepro techno solutions 2014 ieee java project - deadline based resource p...
Ā 
An effective classification approach for big data with parallel generalized H...
An effective classification approach for big data with parallel generalized H...An effective classification approach for big data with parallel generalized H...
An effective classification approach for big data with parallel generalized H...
Ā 
Improving the Performance of Mapping based on Availability- Alert Algorithm U...
Improving the Performance of Mapping based on Availability- Alert Algorithm U...Improving the Performance of Mapping based on Availability- Alert Algorithm U...
Improving the Performance of Mapping based on Availability- Alert Algorithm U...
Ā 
Task Scheduling methodology in cloud computing
Task Scheduling methodology in cloud computing Task Scheduling methodology in cloud computing
Task Scheduling methodology in cloud computing
Ā 
Differentiating Algorithms of Cloud Task Scheduling Based on various Parameters
Differentiating Algorithms of Cloud Task Scheduling Based on various ParametersDifferentiating Algorithms of Cloud Task Scheduling Based on various Parameters
Differentiating Algorithms of Cloud Task Scheduling Based on various Parameters
Ā 
A hybrid approach for scheduling applications in cloud computing environment
A hybrid approach for scheduling applications in cloud computing environment A hybrid approach for scheduling applications in cloud computing environment
A hybrid approach for scheduling applications in cloud computing environment
Ā 
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...
IRJET-Framework for Dynamic Resource Allocation and Efficient Scheduling Stra...
Ā 
REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...
 REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION... REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...
REAL-TIME CHANGE DATA CAPTURE USING STAGING TABLES AND DELTA VIEW GENERATION...
Ā 
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENT
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENTA HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENT
A HYPER-HEURISTIC METHOD FOR SCHEDULING THEJOBS IN CLOUD ENVIRONMENT
Ā 
Scalable scheduling of updates in streaming data warehouses
Scalable scheduling of updates in streaming data warehousesScalable scheduling of updates in streaming data warehouses
Scalable scheduling of updates in streaming data warehouses
Ā 
Berlin Hadoop Get Together Apache Drill
Berlin Hadoop Get Together Apache Drill Berlin Hadoop Get Together Apache Drill
Berlin Hadoop Get Together Apache Drill
Ā 
Role of Operational System Design in Data Warehouse Implementation: Identifyi...
Role of Operational System Design in Data Warehouse Implementation: Identifyi...Role of Operational System Design in Data Warehouse Implementation: Identifyi...
Role of Operational System Design in Data Warehouse Implementation: Identifyi...
Ā 
Resisting skew accumulation
Resisting skew accumulationResisting skew accumulation
Resisting skew accumulation
Ā 
Fault tolerance on cloud computing
Fault tolerance on cloud computingFault tolerance on cloud computing
Fault tolerance on cloud computing
Ā 
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...
Efficient Resource Management Mechanism with Fault Tolerant Model for Computa...
Ā 
A Baye's Theorem Based Node Selection for Load Balancing in Cloud Environment
A Baye's Theorem Based Node Selection for Load Balancing in Cloud EnvironmentA Baye's Theorem Based Node Selection for Load Balancing in Cloud Environment
A Baye's Theorem Based Node Selection for Load Balancing in Cloud Environment
Ā 

Similar to Earlier stage for straggler detection and handling using combined CPU test and LATE methodology

A simulation-based approach for straggler tasks detection in Hadoop MapReduce
A simulation-based approach for straggler tasks detection in Hadoop MapReduceA simulation-based approach for straggler tasks detection in Hadoop MapReduce
A simulation-based approach for straggler tasks detection in Hadoop MapReduceIRJET Journal
Ā 
Scheduling in cloud computing
Scheduling in cloud computingScheduling in cloud computing
Scheduling in cloud computingijccsa
Ā 
Sharing of cluster resources among multiple Workflow Applications
Sharing of cluster resources among multiple Workflow ApplicationsSharing of cluster resources among multiple Workflow Applications
Sharing of cluster resources among multiple Workflow Applicationsijcsit
Ā 
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce Framework
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce FrameworkBIGDATA- Survey on Scheduling Methods in Hadoop MapReduce Framework
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce FrameworkMahantesh Angadi
Ā 
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduceBIGDATA- Survey on Scheduling Methods in Hadoop MapReduce
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduceMahantesh Angadi
Ā 
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...AzarulIkhwan
Ā 
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENT
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENTDYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENT
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENTIJCNCJournal
Ā 
Dynamic Task Scheduling based on Burst Time Requirement for Cloud Environment
Dynamic Task Scheduling based on Burst Time Requirement for Cloud EnvironmentDynamic Task Scheduling based on Burst Time Requirement for Cloud Environment
Dynamic Task Scheduling based on Burst Time Requirement for Cloud EnvironmentIJCNCJournal
Ā 
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORS
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORSMULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORS
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORScscpconf
Ā 
Multiple dag applications
Multiple dag applicationsMultiple dag applications
Multiple dag applicationscsandit
Ā 
Hadoop scheduler with deadline constraint
Hadoop scheduler with deadline constraintHadoop scheduler with deadline constraint
Hadoop scheduler with deadline constraintijccsa
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - a hyper-heuristic sched...
Ieeepro techno solutions   2014 ieee dotnet project - a hyper-heuristic sched...Ieeepro techno solutions   2014 ieee dotnet project - a hyper-heuristic sched...
Ieeepro techno solutions 2014 ieee dotnet project - a hyper-heuristic sched...ASAITHAMBIRAJAA
Ā 
Ieeepro techno solutions 2014 ieee java project - a hyper-heuristic scheduling
Ieeepro techno solutions   2014 ieee java project - a hyper-heuristic schedulingIeeepro techno solutions   2014 ieee java project - a hyper-heuristic scheduling
Ieeepro techno solutions 2014 ieee java project - a hyper-heuristic schedulinghemanthbbc
Ā 
Presented by Ahmed Abdulhakim Al-Absi - Scaling map reduce applications acro...
Presented by Ahmed Abdulhakim Al-Absi -  Scaling map reduce applications acro...Presented by Ahmed Abdulhakim Al-Absi -  Scaling map reduce applications acro...
Presented by Ahmed Abdulhakim Al-Absi - Scaling map reduce applications acro...Absi Ahmed
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...ASAITHAMBIRAJAA
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...ASAITHAMBIRAJAA
Ā 
Distributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data AnalysisDistributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data AnalysisIRJET Journal
Ā 

Similar to Earlier stage for straggler detection and handling using combined CPU test and LATE methodology (20)

A simulation-based approach for straggler tasks detection in Hadoop MapReduce
A simulation-based approach for straggler tasks detection in Hadoop MapReduceA simulation-based approach for straggler tasks detection in Hadoop MapReduce
A simulation-based approach for straggler tasks detection in Hadoop MapReduce
Ā 
Scheduling in cloud computing
Scheduling in cloud computingScheduling in cloud computing
Scheduling in cloud computing
Ā 
Sharing of cluster resources among multiple Workflow Applications
Sharing of cluster resources among multiple Workflow ApplicationsSharing of cluster resources among multiple Workflow Applications
Sharing of cluster resources among multiple Workflow Applications
Ā 
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce Framework
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce FrameworkBIGDATA- Survey on Scheduling Methods in Hadoop MapReduce Framework
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce Framework
Ā 
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduceBIGDATA- Survey on Scheduling Methods in Hadoop MapReduce
BIGDATA- Survey on Scheduling Methods in Hadoop MapReduce
Ā 
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...
Task Scheduling using Tabu Search algorithm in Cloud Computing Environment us...
Ā 
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENT
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENTDYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENT
DYNAMIC TASK SCHEDULING BASED ON BURST TIME REQUIREMENT FOR CLOUD ENVIRONMENT
Ā 
Dynamic Task Scheduling based on Burst Time Requirement for Cloud Environment
Dynamic Task Scheduling based on Burst Time Requirement for Cloud EnvironmentDynamic Task Scheduling based on Burst Time Requirement for Cloud Environment
Dynamic Task Scheduling based on Burst Time Requirement for Cloud Environment
Ā 
C1803052327
C1803052327C1803052327
C1803052327
Ā 
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORS
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORSMULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORS
MULTIPLE DAG APPLICATIONS SCHEDULING ON A CLUSTER OF PROCESSORS
Ā 
Multiple dag applications
Multiple dag applicationsMultiple dag applications
Multiple dag applications
Ā 
Hadoop scheduler with deadline constraint
Hadoop scheduler with deadline constraintHadoop scheduler with deadline constraint
Hadoop scheduler with deadline constraint
Ā 
C044051215
C044051215C044051215
C044051215
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - a hyper-heuristic sched...
Ieeepro techno solutions   2014 ieee dotnet project - a hyper-heuristic sched...Ieeepro techno solutions   2014 ieee dotnet project - a hyper-heuristic sched...
Ieeepro techno solutions 2014 ieee dotnet project - a hyper-heuristic sched...
Ā 
Ieeepro techno solutions 2014 ieee java project - a hyper-heuristic scheduling
Ieeepro techno solutions   2014 ieee java project - a hyper-heuristic schedulingIeeepro techno solutions   2014 ieee java project - a hyper-heuristic scheduling
Ieeepro techno solutions 2014 ieee java project - a hyper-heuristic scheduling
Ā 
335 340
335 340335 340
335 340
Ā 
Presented by Ahmed Abdulhakim Al-Absi - Scaling map reduce applications acro...
Presented by Ahmed Abdulhakim Al-Absi -  Scaling map reduce applications acro...Presented by Ahmed Abdulhakim Al-Absi -  Scaling map reduce applications acro...
Presented by Ahmed Abdulhakim Al-Absi - Scaling map reduce applications acro...
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ā 
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...Ieeepro techno solutions   2014 ieee dotnet project - deadline based resource...
Ieeepro techno solutions 2014 ieee dotnet project - deadline based resource...
Ā 
Distributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data AnalysisDistributed Feature Selection for Efficient Economic Big Data Analysis
Distributed Feature Selection for Efficient Economic Big Data Analysis
Ā 

More from IJECEIAES

Cloud service ranking with an integration of k-means algorithm and decision-m...
Cloud service ranking with an integration of k-means algorithm and decision-m...Cloud service ranking with an integration of k-means algorithm and decision-m...
Cloud service ranking with an integration of k-means algorithm and decision-m...IJECEIAES
Ā 
Prediction of the risk of developing heart disease using logistic regression
Prediction of the risk of developing heart disease using logistic regressionPrediction of the risk of developing heart disease using logistic regression
Prediction of the risk of developing heart disease using logistic regressionIJECEIAES
Ā 
Predictive analysis of terrorist activities in Thailand's Southern provinces:...
Predictive analysis of terrorist activities in Thailand's Southern provinces:...Predictive analysis of terrorist activities in Thailand's Southern provinces:...
Predictive analysis of terrorist activities in Thailand's Southern provinces:...IJECEIAES
Ā 
Optimal model of vehicular ad-hoc network assisted by unmanned aerial vehicl...
Optimal model of vehicular ad-hoc network assisted by  unmanned aerial vehicl...Optimal model of vehicular ad-hoc network assisted by  unmanned aerial vehicl...
Optimal model of vehicular ad-hoc network assisted by unmanned aerial vehicl...IJECEIAES
Ā 
Improving cyberbullying detection through multi-level machine learning
Improving cyberbullying detection through multi-level machine learningImproving cyberbullying detection through multi-level machine learning
Improving cyberbullying detection through multi-level machine learningIJECEIAES
Ā 
Comparison of time series temperature prediction with autoregressive integrat...
Comparison of time series temperature prediction with autoregressive integrat...Comparison of time series temperature prediction with autoregressive integrat...
Comparison of time series temperature prediction with autoregressive integrat...IJECEIAES
Ā 
Strengthening data integrity in academic document recording with blockchain a...
Strengthening data integrity in academic document recording with blockchain a...Strengthening data integrity in academic document recording with blockchain a...
Strengthening data integrity in academic document recording with blockchain a...IJECEIAES
Ā 
Design of storage benchmark kit framework for supporting the file storage ret...
Design of storage benchmark kit framework for supporting the file storage ret...Design of storage benchmark kit framework for supporting the file storage ret...
Design of storage benchmark kit framework for supporting the file storage ret...IJECEIAES
Ā 
Detection of diseases in rice leaf using convolutional neural network with tr...
Detection of diseases in rice leaf using convolutional neural network with tr...Detection of diseases in rice leaf using convolutional neural network with tr...
Detection of diseases in rice leaf using convolutional neural network with tr...IJECEIAES
Ā 
A systematic review of in-memory database over multi-tenancy
A systematic review of in-memory database over multi-tenancyA systematic review of in-memory database over multi-tenancy
A systematic review of in-memory database over multi-tenancyIJECEIAES
Ā 
Agriculture crop yield prediction using inertia based cat swarm optimization
Agriculture crop yield prediction using inertia based cat swarm optimizationAgriculture crop yield prediction using inertia based cat swarm optimization
Agriculture crop yield prediction using inertia based cat swarm optimizationIJECEIAES
Ā 
Three layer hybrid learning to improve intrusion detection system performance
Three layer hybrid learning to improve intrusion detection system performanceThree layer hybrid learning to improve intrusion detection system performance
Three layer hybrid learning to improve intrusion detection system performanceIJECEIAES
Ā 
Non-binary codes approach on the performance of short-packet full-duplex tran...
Non-binary codes approach on the performance of short-packet full-duplex tran...Non-binary codes approach on the performance of short-packet full-duplex tran...
Non-binary codes approach on the performance of short-packet full-duplex tran...IJECEIAES
Ā 
Improved design and performance of the global rectenna system for wireless po...
Improved design and performance of the global rectenna system for wireless po...Improved design and performance of the global rectenna system for wireless po...
Improved design and performance of the global rectenna system for wireless po...IJECEIAES
Ā 
Advanced hybrid algorithms for precise multipath channel estimation in next-g...
Advanced hybrid algorithms for precise multipath channel estimation in next-g...Advanced hybrid algorithms for precise multipath channel estimation in next-g...
Advanced hybrid algorithms for precise multipath channel estimation in next-g...IJECEIAES
Ā 
Performance analysis of 2D optical code division multiple access through unde...
Performance analysis of 2D optical code division multiple access through unde...Performance analysis of 2D optical code division multiple access through unde...
Performance analysis of 2D optical code division multiple access through unde...IJECEIAES
Ā 
On performance analysis of non-orthogonal multiple access downlink for cellul...
On performance analysis of non-orthogonal multiple access downlink for cellul...On performance analysis of non-orthogonal multiple access downlink for cellul...
On performance analysis of non-orthogonal multiple access downlink for cellul...IJECEIAES
Ā 
Phase delay through slot-line beam switching microstrip patch array antenna d...
Phase delay through slot-line beam switching microstrip patch array antenna d...Phase delay through slot-line beam switching microstrip patch array antenna d...
Phase delay through slot-line beam switching microstrip patch array antenna d...IJECEIAES
Ā 
A simple feed orthogonal excitation X-band dual circular polarized microstrip...
A simple feed orthogonal excitation X-band dual circular polarized microstrip...A simple feed orthogonal excitation X-band dual circular polarized microstrip...
A simple feed orthogonal excitation X-band dual circular polarized microstrip...IJECEIAES
Ā 
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...A taxonomy on power optimization techniques for fifthgeneration heterogenous ...
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...IJECEIAES
Ā 

More from IJECEIAES (20)

Cloud service ranking with an integration of k-means algorithm and decision-m...
Cloud service ranking with an integration of k-means algorithm and decision-m...Cloud service ranking with an integration of k-means algorithm and decision-m...
Cloud service ranking with an integration of k-means algorithm and decision-m...
Ā 
Prediction of the risk of developing heart disease using logistic regression
Prediction of the risk of developing heart disease using logistic regressionPrediction of the risk of developing heart disease using logistic regression
Prediction of the risk of developing heart disease using logistic regression
Ā 
Predictive analysis of terrorist activities in Thailand's Southern provinces:...
Predictive analysis of terrorist activities in Thailand's Southern provinces:...Predictive analysis of terrorist activities in Thailand's Southern provinces:...
Predictive analysis of terrorist activities in Thailand's Southern provinces:...
Ā 
Optimal model of vehicular ad-hoc network assisted by unmanned aerial vehicl...
Optimal model of vehicular ad-hoc network assisted by  unmanned aerial vehicl...Optimal model of vehicular ad-hoc network assisted by  unmanned aerial vehicl...
Optimal model of vehicular ad-hoc network assisted by unmanned aerial vehicl...
Ā 
Improving cyberbullying detection through multi-level machine learning
Improving cyberbullying detection through multi-level machine learningImproving cyberbullying detection through multi-level machine learning
Improving cyberbullying detection through multi-level machine learning
Ā 
Comparison of time series temperature prediction with autoregressive integrat...
Comparison of time series temperature prediction with autoregressive integrat...Comparison of time series temperature prediction with autoregressive integrat...
Comparison of time series temperature prediction with autoregressive integrat...
Ā 
Strengthening data integrity in academic document recording with blockchain a...
Strengthening data integrity in academic document recording with blockchain a...Strengthening data integrity in academic document recording with blockchain a...
Strengthening data integrity in academic document recording with blockchain a...
Ā 
Design of storage benchmark kit framework for supporting the file storage ret...
Design of storage benchmark kit framework for supporting the file storage ret...Design of storage benchmark kit framework for supporting the file storage ret...
Design of storage benchmark kit framework for supporting the file storage ret...
Ā 
Detection of diseases in rice leaf using convolutional neural network with tr...
Detection of diseases in rice leaf using convolutional neural network with tr...Detection of diseases in rice leaf using convolutional neural network with tr...
Detection of diseases in rice leaf using convolutional neural network with tr...
Ā 
A systematic review of in-memory database over multi-tenancy
A systematic review of in-memory database over multi-tenancyA systematic review of in-memory database over multi-tenancy
A systematic review of in-memory database over multi-tenancy
Ā 
Agriculture crop yield prediction using inertia based cat swarm optimization
Agriculture crop yield prediction using inertia based cat swarm optimizationAgriculture crop yield prediction using inertia based cat swarm optimization
Agriculture crop yield prediction using inertia based cat swarm optimization
Ā 
Three layer hybrid learning to improve intrusion detection system performance
Three layer hybrid learning to improve intrusion detection system performanceThree layer hybrid learning to improve intrusion detection system performance
Three layer hybrid learning to improve intrusion detection system performance
Ā 
Non-binary codes approach on the performance of short-packet full-duplex tran...
Non-binary codes approach on the performance of short-packet full-duplex tran...Non-binary codes approach on the performance of short-packet full-duplex tran...
Non-binary codes approach on the performance of short-packet full-duplex tran...
Ā 
Improved design and performance of the global rectenna system for wireless po...
Improved design and performance of the global rectenna system for wireless po...Improved design and performance of the global rectenna system for wireless po...
Improved design and performance of the global rectenna system for wireless po...
Ā 
Advanced hybrid algorithms for precise multipath channel estimation in next-g...
Advanced hybrid algorithms for precise multipath channel estimation in next-g...Advanced hybrid algorithms for precise multipath channel estimation in next-g...
Advanced hybrid algorithms for precise multipath channel estimation in next-g...
Ā 
Performance analysis of 2D optical code division multiple access through unde...
Performance analysis of 2D optical code division multiple access through unde...Performance analysis of 2D optical code division multiple access through unde...
Performance analysis of 2D optical code division multiple access through unde...
Ā 
On performance analysis of non-orthogonal multiple access downlink for cellul...
On performance analysis of non-orthogonal multiple access downlink for cellul...On performance analysis of non-orthogonal multiple access downlink for cellul...
On performance analysis of non-orthogonal multiple access downlink for cellul...
Ā 
Phase delay through slot-line beam switching microstrip patch array antenna d...
Phase delay through slot-line beam switching microstrip patch array antenna d...Phase delay through slot-line beam switching microstrip patch array antenna d...
Phase delay through slot-line beam switching microstrip patch array antenna d...
Ā 
A simple feed orthogonal excitation X-band dual circular polarized microstrip...
A simple feed orthogonal excitation X-band dual circular polarized microstrip...A simple feed orthogonal excitation X-band dual circular polarized microstrip...
A simple feed orthogonal excitation X-band dual circular polarized microstrip...
Ā 
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...A taxonomy on power optimization techniques for fifthgeneration heterogenous ...
A taxonomy on power optimization techniques for fifthgeneration heterogenous ...
Ā 

Recently uploaded

(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escortsranjana rawat
Ā 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxupamatechverse
Ā 
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
Ā 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024hassan khalil
Ā 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
Ā 
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”soniya singh
Ā 
Analog to Digital and Digital to Analog Converter
Analog to Digital and Digital to Analog ConverterAnalog to Digital and Digital to Analog Converter
Analog to Digital and Digital to Analog ConverterAbhinavSharma374939
Ā 
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICSAPPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICSKurinjimalarL3
Ā 
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).pptssuser5c9d4b1
Ā 
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...Dr.Costas Sachpazis
Ā 
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptxWhat are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptxwendy cai
Ā 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escortsranjana rawat
Ā 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxAsutosh Ranjan
Ā 
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130Suhani Kapoor
Ā 
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxDecoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxJoĆ£o Esperancinha
Ā 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
Ā 
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube Exchanger
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube ExchangerStudy on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube Exchanger
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube ExchangerAnamika Sarkar
Ā 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSRajkumarAkumalla
Ā 

Recently uploaded (20)

(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
Ā 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptx
Ā 
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCRCall Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Call Us -/9953056974- Call Girls In Vikaspuri-/- Delhi NCR
Ā 
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
Ā 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024
Ā 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
Ā 
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”
Model Call Girl in Narela Delhi reach out to us at šŸ”8264348440šŸ”
Ā 
Analog to Digital and Digital to Analog Converter
Analog to Digital and Digital to Analog ConverterAnalog to Digital and Digital to Analog Converter
Analog to Digital and Digital to Analog Converter
Ā 
9953056974 Call Girls In South Ex, Escorts (Delhi) NCR.pdf
9953056974 Call Girls In South Ex, Escorts (Delhi) NCR.pdf9953056974 Call Girls In South Ex, Escorts (Delhi) NCR.pdf
9953056974 Call Girls In South Ex, Escorts (Delhi) NCR.pdf
Ā 
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICSAPPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
APPLICATIONS-AC/DC DRIVES-OPERATING CHARACTERISTICS
Ā 
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
Ā 
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Ā 
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptxWhat are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
Ā 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
Ā 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptx
Ā 
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130
VIP Call Girls Service Kondapur Hyderabad Call +91-8250192130
Ā 
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxDecoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Ā 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Ā 
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube Exchanger
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube ExchangerStudy on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube Exchanger
Study on Air-Water & Water-Water Heat Exchange in a Finned ļ»æTube Exchanger
Ā 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
Ā 

Earlier stage for straggler detection and handling using combined CPU test and LATE methodology

  • 1. International Journal of Electrical and Computer Engineering (IJECE) Vol. 10, No. 5, October 2020, pp. 4910~4917 ISSN: 2088-8708, DOI: 10.11591/ijece.v10i5.pp4910-4917 ļ² 4910 Journal homepage: http://ijece.iaescore.com/index.php/IJECE Earlier stage for straggler detection and handling using combined CPU test and LATE methodology Anwar H. Katrawi1 , Rosni Abdullah2 , Mohammed Anbar3 , Ammar Kamal Abasi4 1,3 National Advanced IPv6 Center (Nav6), Universiti Sains Malaysia, Malaysia 2,4 School of Computer Sciences, Universiti Sains Malaysia, Malaysia Article Info ABSTRACT Article history: Received Oct 15, 2019 Revised Mar 17, 2020 Accepted Mar 30, 2020 Using MapReduce in Hadoop helps in lowering the execution time and power consumption for large scale data. However, there can be a delay in job processing in circumstances where tasks are assigned to bad or congested machines called "straggler tasks"; which increases the time, power consumptions and therefore increasing the costs and leading to a poor performance of computing systems. This research proposes a hybrid MapReduce framework referred to as the combinatory late-machine (CLM) framework. Implementation of this framework will facilitate early and timely detection and identification of stragglers thereby facilitating prompt appropriate and effective actions. Keywords: Big data Combinatory late-machine Hadoop Map reduce Straggler Copyright Ā© 2020 Institute of Advanced Engineering and Science. All rights reserved. Corresponding Author: Anwar H. Katrawi, National Advanced IPv6 Center (Nav6), Universiti Sains Malaysia, 11800 USM, Penang, Malaysia. Email: akatrawi@student.usm.my 1. INTRODUCTION A significant amount of data (big data) is stored and transferred online by thousands of companies, organizations, and individuals. This largely unstructured data is difficult to analyze and process using conventional database management tools which creates new challenges in the analysis and the storage of data [1]. Recently, there has been an increasing interest in key areas such as real-time data extraction, which reveals an urgent need for bulk and strict performance constraints. Consequently, the adaptation of huge data to implementations on distributed computing platforms is necessary. One way of doing this is to adopt and implement the popular programming model known as MapReduce [2]. Its success lies in simplicity, scalability, efficiency and extensibility that pushes the IT industry leaders such as Google, Yahoo, Facebook and Amazon to extensively adopt MapReduce as a powerful and reliable tool for Big Data processing. There are four factors, including processing, storing, visualization, and analyzing large data in modern organizations and enterprises. MapReduce can run the applications on a parallel cluster of hardware automatically. In addition; it can process terabytes and petabytes of data more rapidly [3,4]. Recently, it has gained popularity in a wide range of applications due to its ability to provide a highly effective and efficient framework for the parallel execution of the applications, data allocation in distributed database systems, and fault tolerance network communications [5]. For instance, Google runs more than 10,000 distinct programs using MapReduce including graph processing[6], text processing, machine learning, and statistical machine translation. Moreover, the famous open-source Hadoop software framework for distributed storage and processing of big data Figure 1 sets uses Mapreduce as a central tool to split the data, process it and make it not only manageable but also available for usersā€™ consumption or further processing. Also, the Hadoop MapReduce environment provides fault-tolerant solutions in case of hardware failures or software errors during the execution of tasks [7].
  • 2. Int J Elec & Comp Eng ISSN: 2088-8708 ļ² Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi) 4911 Figure 1. Hadoop framework According to the work presented by [8], Hadoop MapReduce has the ability to tolerate several types of faults and they are as follows: a. Nodes failure: A node in a MapReduce cluster may fail at any time. In this case, the JobTracker removes this slave node from the list of nodes available and re-executes the tasks on other nodes. It can be concluded that a node is declared failing if at least one task launched on it has failed. At the time, the JobTracker checks if the node in question should not be blacklisted. If a slave node is "blacklisted", the JobTracker will no longer assign it map or reduce. It can be removed from this list if his behavior becomes normal and does not commit faults during a certain time interval. b. Software Failure: A task may stop because of an error or exception in the mapping or reduction program. In this case, the JobTracker orders the re-execution of the failed task to a limited number of attempts (four by default), beyond which the task and the job of the task is considered faulty. c. Stopped Task: As an example, the process of running a task may stop unexpectedly due to a transient bug in the underlying virtual machine. In this case, the JobTracker show in Figure 2 will be notified and it will reorder the job as described above. d. Blocked Task: Itā€™s considered faulty if after some timea mapping or reduction task remains blocked without any progress; in this case, the JobTracker orders to kill the process running this task. e. Delayed Tasks:When some tasks are unexpectedly taking longer execution time compared with the average execution time, these tasks called stragglers. Figure 2. Job tracker Several studies have been devoted to improving the tolerance of systems to faults. For instance, Microsoft reveals that when the CPU and core partsā€™ errors occur in one million customersā€™ computers, Hadoop does not have the capacity to deal with these types of potential errors. This also includes other types of errors related to the task itself (when one or more of these tasks take longer or stops before the work outcomes are realized). Problem statement. The delayed tasks are called stragglers and play a key role in increasing the execution time of big data and energy consumption. Following the MapReduce framework, stragglers refer to the tasks that take a longer time to be executed compared to other tasks. There are various techniques of detecting and handling stragglers such as Dolly, the Hadoop native scheduler, MonTool, LATE and Mantri. Regardless of the techniques already in place, straggler detection remains problematic in the field of data analytics.
  • 3. ļ² ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917 4912 The proposed solution. We propose an algorithm that facilitates the calculation of straggler tolerance threshold using CPU test and LATE methodology. Our approach considers crucial issues such as the QoS with a major focus on the timing constraints, the progress of task execution, and the usage of cluster resources. Even though task selection does not appear to be a big problem in the initial stages, we have been able to show that it is a big issue that requires close attention. Therefore, we recommend identifying those tasks that lead to the longest response times. We also recommend that this needs to be done as early as possible so that there could be no later surprises. Taking these into account, it is possible to see that our LATE methodology is based on the estimate of the time that is left with the goal of early detection of the tasks that are running slowly. In summary, the methodology is based on making decisions early, using the finishing times and not the progress rates, not assigning speculative tasks to slow nodes and optimization of resource utilization. Significance of the research. In the current study, a method is proposed with the purpose of addressing the straggler problem. We propose an algorithm which can enable calculation of threshold of straggler tolerance earlier using CPU test and Longest Approximate Time to End (LATE) methodology. Our approach focuses on the timing constraints, progress of task execution, and the usage of cluster resources. In this work, our major contribution is that unlike other studies that assume that it is hard to have a correlation between task execution and node status, we show that it is possible and feasible to have a correlation and detect stragglers using LATE methodology and CPU test. Our methodology is simple enough and easy to accommodate with low overheads. Organization of the research. The rest of this paper is organized as follows: Section 2 covers the literature review, section 3 covers the methodology, section 4 the results and discussion, section 5 covers the proposed solution/recommendation, and section 6 covers the conclusion. 2. LITERATURE REVIEW Hadoop and Map Reduce are among the most commonly used frameworks when it comes to task execution across several nodes for optimal performance. Even though the frameworks have become popular, they still face several challenges when it comes to the effectiveness of tasks execution [8]. Specifically, achieving predictable execution has become problematic because of stragglers. Due to stragglers, tasks execution takes longer to complete than originally anticipated [9]. Such delays are undesirable because they result in reduced service performance and can also potentially violate QoS (Quality of Service) requirements concerning time taken to complete tasks. For service providers, tasks that take more time to complete lead to reduced availability of systems and cause jobs to consume more time. Stragglers have become common especially in cloud data centers [10]. Therefore, it is vital to detect and mitigate them promptly. Additionally, centers with large computing infrastructure can also experience delays that can lead to ineffective job execution. Also, large data centers have a high intake of service creation which make them vulnerable to stragglers. There are several root causes to stragglers including resource contention, hardware heterogeneity, background network traffic, and operating system related-level causes [11]. Considerable effort has been made to study stragglers. Over the years, the size of computing infrastructure and jobs executed have continued to grow which has dramatically increased the impact of stragglers. Stragglers are known to extend job execution substantially which negatively affects the ā€œConsumer Service Level Agreementā€ and QoS performance requirements. In a study conducted by Bortnikov, Frank, Hillel, and Rao, the authors propose two ways of dealing with stragglers namely tolerance and avoidance [12]. However, avoiding stragglers is difficult since it is impractical to pursue. Therefore, straggler tolerance is the approach adopted by most stakeholders. In straggler tolerance, the execution progress of a task is monitored using a percentage score made up of values ranging from 0 to 1 which represent start and completion. Currently, the approaches used for straggler detection can either be described as offline or online analytics [13]. Nonetheless, it is worth noting that online detection can occur too late during the execution cycle of a task. Therefore, stragglers cannot be prevented from running slower even after the implementation of speculative copies. On the other hand, offline approaches are normally applied to avoid stragglers. This approach is seen as less feasible and thus it is uncommon. However, better results can be achieved by combining both online and offline approaches. When used together, they can significantly help to improve the effectiveness of ā€œstraggler detectionā€. 2.1. Related works For Straggler Detection, many techniques have been developed. Ouyang et al. proposes an algorithm based on the progress score of task execution that enables dynamic threshold detection for straggler tasks [14]. This strategy has improved performance significantly by reducing job execution by 44 percent.
  • 4. Int J Elec & Comp Eng ISSN: 2088-8708 ļ² Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi) 4913 In a study conducted by Zaharia et al., the authors propose a new Straggler Detection that takes into account both the progress score and elapsed time with the objective of improving the progress score strategy [15]. The authors determine that the strategy does not have the capacity of determining how fast a task runs among different tasks starting at different times hence the need for improvement. Dean and Ghemawat have adopted a technique called Speculative Execution in which they launch copies of the straggler on alternative nodes with the aim of improving performance [16]. Google acknowledges that Speculative Execution improves job execution by 44 percent. However, this technique can reduce the overall throughput due to the duplication of tasks. Therefore, some Hadoop administrators prefer not to use the Speculative Execution option [17, 18]. Yanfei et al proposed another technique for Straggler handling which consists of ending the delayed tasks and reassigning them to another node without stragglers, however, Guo, Rao, Jiang, and Zhou disagrees with this technique and argue that it results in wastage of resources and therefore increases energy consumption [19]. In a similar study, Zhou, Li, Yang, Jia, and Li propose a technique known as ā€œBigRootsā€ which involves incorporating both system features and framework for the analysis of root causes of stragglers especially in big data systems [20]. The authors establish that ā€œBigRootsā€ is effective when it comes to identifying the ā€œroot causesā€ of stragglers which can significantly help in optimizing performance. An examination by Phan attempted to come up with ā€œenergy efficient straggler mitigationā€ technique for big data applications especially in the cloud environment [21-23]. The framework employed by the author takes into account how heterogeneity of resources affect the performance and energy consumption of big data applications. In another studies by Harlap et al., Kim, W., the authors sought to solve the straggler problem for parallel ML [24, 25]. The authors combined a more ā€œflexible synchronization modelā€ together with the experiments involving real straggler behaviors and synthetic straggler behaviors to come up with near-ideal run times across all the straggler patterns they tested. Similarly, Yadwadkar et al. came up with a framework called ā€œWranglerā€ which could predict when stragglers were going to occur and aid in making scheduling decisions [26]. The formulations employed by the authors captured the shared structure in their data so that it could improve the generalization performance of their data. 3. METHODOLOGY As noted earlier, we estimate the time remaining for each task based on the process score derived from Hadoop. In practice, this heuristic works well. However, we want to point out that there are incidences when it can backfire. When this happens, the heuristic can provide incorrect estimates and give results that a task launched later finishes earlier. To demonstrate the delay, we assume that the progress of a task grows by five percent during the first phase. We assume that during this first phase, the total score is fifty percent and that the rate reduces by one percent in the second phase. In the first phase, it is expected that the task will take ten seconds and fifty seconds in the second phase to produce a total of sixty seconds. When two copies of the same task are launched at the same time, the first task is denoted by T1, the second is noted by T2 and the first task starts at time 0 while the next starts after ten seconds. The progress rate is checked after twenty seconds whereby twenty seconds, it is expected that T1 will have finished the first phase and will be through a fifth of the second phase. Therefore, it will have a progress score of sixty percent. Its rate of progress will be 60%/20s=3%/s. On the other hand, T2 will just be through with the first and its score will be 50%. Its rate will, therefore, be 50%/10s=5%/s. The estimated time remaining for T1 will be (100%-60%)/(3%/s)=13.3s. For T2, the estimated time left will be (100%-50%)/(5%/s)=10s. Therefore, the heuristic will illustrate that T1 will take a longer time to run compared to T2. However, in reality, T2 will finish second compared to T1. We also determined the criteria that we could use to identify stragglers. We took the finish time for every task to be represented by (1). EFš‘‡ š‘– š‘— =tk + 1āˆ’PS(task š‘– š‘— ) PS(task š‘– š‘— ) (tk-t0) (1) In this instance, EFš‘‡ š‘– š‘— represents the estimated finish time. PS is used to represent the progress score for a š‘”ask š‘– š‘— whereas t0 is starting time while tk is the timestamp recording for PS(task š‘– š‘— ). Our proposed LATE methodology takes into account the scope of data and the speed of processing data. By taking these into account, we can determine the pattern of straggler detection and correlate this with the attributes of a system normally hypothesized to give them the ability to cause stragglers. Some of the attributes include resource utilization (memory, CPU, disk), hardware faults, unhandled requests, among others. When we
  • 5. ļ² ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917 4914 combine these runtime features with dynamic information, we can come up with better prediction and learning models that can discriminate stragglers promptly and lower the number of failures occurring due to late timing. 4. RESULTS AND DISCUSSION This research has identified the need for improving the effectiveness of MapReduce processes to facilitate costs reduction and maximum utilization of resources. Performance improvement can be achieved by eliminating inefficiencies brought about by the existence of stragglers. Stragglers have the effect of resulting in poor user code and uneven aggregation of workloads. Poor user code is the product of looping conditions that are designed ineffectively and uneven aggregation of workloads results from extreme co-allocation of workloads due to inefficient scheduling. In the case of MapReduce tasks with a massive number of write and read queries, it is common for file requests to be overloaded thereby leading to inefficiency in handling the requests. It is notable that once the threshold of the master node is surpassed, stragglers set in which implies that the waiting queue for requests becomes long. If the requests continue increasing, then the master node becomes overloaded thereby further slowing down the handling process of the requests. This experiment focused on investigating how the occurrence of stragglers is affected by contention of resources. An observation of the occurrence of stragglers that took place over a period of 20 days yielded the results presented in Figure 3. Figure 3. Graphical presentation of the findings In the analysis, millions of tasks processed in five hundred servers in Cloud Datacenter [27] were investigated. The inclusion criteria involved: a. Servers whose tasks had DoS-Index values greater than or equal to 10 b. Servers whose utilization of CPU was greater than or equal to 80% c. Latency from file processing greater than 400ms which translates to slow handling of write and read requests From the investigation, it came to our attention that 42% of stragglers are brought about by overloading of disks while 59% of stragglers exist under high server CPU conditions. The findings also revealed that slow handling of requests was responsible for the occurrence of 34.3% of stragglers. From the findings, it is evident that the existence of stragglers is significantly caused by high utilization of resources. It was also observed that stragglers can be caused by other factors such as the conditions of the network. 5. RECOMMENDATION/PROPOSED SOLUTION In this paper, we propose a combined strategy called COMBINATORY LATE-MACHINE that tests the CPU machine. The CPU machine is tested to determine its vulnerability to stragglers. To achieve this, the CPU and the RAM are tested to determine machines that have a CPU usage less than 85. Those found to meet this condition are dropped and the ones that exceed 85 are selected and placed in a performance chart
  • 6. Int J Elec & Comp Eng ISSN: 2088-8708 ļ² Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi) 4915 starting with those that have the highest performance. If for any reason the CPU/Memory is lower than a certain threshold, the jobs will automatically be redirected to another machine with a higher CPU/Memory performance. This stage is run once and helps in creating a list of performances for all the machines. After this step, Straggler Detection and handling are initiated at the same time using LATE algorithm. The algorithm has several benefits that make it suitable for heterogeneous jobs since it re-executes only the slowest tasks. The primary advantage of the proposed methodology is the increasing probability to detect the straggler machines and straggler tasks with the same algorithm in earlier stages. Our LATE scheduler is designed such that it includes all the features needed for it to function well in a realistic environment. The major insight behind the LATE algorithm is that tasks believed to finish last can be executed at any time in the future because this is the best way through which the response time can be improved. The framework of the proposed combinatory Late-Machine strategy is illustrated in Figure 4. Figure 4. Proposed combinatory late-machine framework 5.1. Limitations and future research Our observation did not focus on the conditions of the memory capacity as well as overlapping of conditions. Therefore, this warrants the need for further investigations. 6. CONCLUSION In this paper, we proposed the Combinatory Late-Machine (CLM) strategy to identify (in earlier stages) the straggler for both nodes and tasks, and the optimal node for re-execution of slow tasks. The overall execution time is improved significantly using this technique compared to traditional job schedulers. In the future, the CLM strategy will be tested with Hadoop to evaluate the efficiency of this technique. The findings provide new insights into early straggler detection. However, the current strategy has its shortcomings and there is a need for exhaustive analytics to establish the relationship between stragglers and the contention of resources. The proposed algorithm could be used for research as well as in the industry to improve the time and cost for big data processing. ACKNOWLEDGMENT The authors would like to thank all the research participants for their time, effort, and contribution to the research
  • 7. ļ² ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 10, No. 5, October 2020 : 4910 - 4917 4916 REFERENCES [1] E. A. Mohammed, et al., ā€œApplications of the MapReduce programming framework to clinical big data analysis: current landscape and future trends,ā€ Bio Data Mining, vol. 7, no. 1, pp. 22-44, 2014. [2] S. ValvĆ„g, et al., ā€œCogset: a high performance MapReduce engine,ā€ Concurrency and Computation: Practice and Experience, vol. 25, no. 1, pp. 2-23, 2012. [3] Sumalatha, S., and Subramanyam, R. B. V. ā€œDistributed mining of high utility time interval sequential patterns using mapreduce approach,ā€ Expert Systems with Applications, vol. 141, 112967, Mar. 2020. [4] Meddah, I. H., and Belkadi, K. ā€œParallel Distributed Patterns Mining Using Hadoop MapReduce Framework,ā€ International Journal of Grid and High Performance Computing, vol. 9, no. 2, pp. 70-85, 2017. [5] S. Khezr and N. J. Navimipour, ā€œMapReduce and Its Applications, Challenges, and Architecture: a Comprehensive Review and Directions for Future Research,ā€ Journal of Grid Computing, vol. 15, no. 3, pp. 295-321, 2017. [6] Prameela De'vi. Chillakuru, T. Kumanan, CH. Sarada Devi, ā€œContent based Retrieval Management Systems in Web Engineering,ā€ International Journal of Recent Technology and Engineering (IJRTE), vol. 8, no. 2S11, pp. 81-93, Sep. 2019 [7] A. Y. Pigul, ā€œComparative Study Parallel Join Algorithms for MapReduce environment,ā€ Proceedings of the Institute for System Programming of RAS, vol. 23, pp. 285-306, 2012. [8] I. Hashem, et al., ā€œMapReduce scheduling algorithms: a review,ā€ The Journal of Supercomputing, 2018. [9] K. Mitsuzuka, et al., ā€œProxy Responses by FPGA-Based Switch for MapReduce Stragglers,ā€ IEICE Transactions on Information and Systems, vol. 101, no. 9, pp. 2258-2268, 2018. [10] J. Rogoff, ā€œStragglers,ā€ Sewanee Review, vol. 124, no. 3, pp. 397-397, 2016. [11] M. F. Aktas, et al., ā€œStraggler Mitigation by Delayed Relaunch of Tasks,ā€ ACM SIGMETRICS Performance Evaluation Review, vol. 45, no. 2, pp. 248-248, 2018. [12] E. Bortnikov, et al., ā€œPredicting execution bottlenecks in map-reduce clusters,ā€ Proceedings of the 4th USENIX conference on Hot Topics in Cloud Computing, pp. 1-18, 2012. [13] A. K. Abasi, et al., ā€œLink-based multi-verse optimizer for text documents clustering,ā€ Applied Soft Computing, vol. 87, 2019. [14] X. Ouyang, et al., ā€œStraggler Detection in Parallel Computing Systems through Dynamic Threshold Calculation,ā€ 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pp. 414-421, 2016. [15] J. Xie, et al., ā€œImproving MapReduce performance through data placement in heterogeneous Hadoop clusters,ā€ 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), pp. 1-9, 2010. [16] J. Dean and S. Ghemawat, ā€œMap Reduce: simplified data processing on large clusters,ā€ Communications of the ACM, vol. 51, no. 1, p. 107, 2008. [17] H. Wu, et al., ā€œA Heuristic Speculative Execution Strategy in Heterogeneous Distributed Environments,ā€ 2014 Sixth International Symposium on Parallel Architectures, Algorithms and Programming, pp. 268-273, 2014. [18] A. K. Abasi, et al., ā€œA Text Feature Selection Technique based on Binary Multi-Verse Optimizer for Text Clustering,ā€ 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), Amman, Jordan, pp. 1-6, 2019. [19] Y. Guo, et al., ā€œFlexSlot: Moving Hadoop Into the Cloud with Flexible Slot Management,ā€ SC14: International Conference for High Performance Computing, Networking, Storage and Analysis, 2014. [20] H. Zhou, et al., ā€œBigRoots: An Effective Approach for Root-Cause Analysis of Stragglers in Big Data System,ā€ IEEE Access, vol. 6, pp. 41966-41977, 2018. [21] T. Phan, et al., ā€œA New Framework for Evaluating Straggler Detection Mechanisms in MapReduce,ā€ ACM Transactions on Modeling and Performance Evaluation of Computing Systems, vol. 4, no. 3, pp. 1-23, 2019. [22] Chaowei Yang, Qunying Huang, Zhenlong Li, Kai Liu & Fei Hu, ā€œBig Data and cloud computing: innovation opportunities and challenges,ā€ International Journal of Digital Earth, vol. 10, no. 1, pp. 13-53, 2017. DOI: 10.1080/17538947.2016.1239771 [23] L. Greeshma, Pradeepini Gera, ā€œBig Data Analytics with Apache Hadoop MapReduce Framework,ā€ Indian Journal of Science and Technology, Vol 9, no. 26, July 2016. [24] A. Harlap, et al., ā€œAddressing the straggler problem for iterative convergent parallel ML,ā€ Proceedings of the Seventh ACM Symposium on Cloud Computing - SoCC '16, pp. 98-111, 2016. [25] Kim, W., Kim, Y., and Shim, K., ā€œParallel computation of k-nearest neighbor joins using MapRe-duce,ā€ In Proceedings of the IEEE International Conference on Big Data, pp. 696-705, 2016. [26] N. Yadwadkar, et al., ā€œMulti-Task Learning for Straggler Avoiding Predictive Job Scheduling,ā€ Journal of Machine Learning Research, vol. 17, pp. 1-37, 2016. [Online]. Available: http://jmlr.org/papers/volume17/15-149/15-149.pdf. [27] Gigas the cloud computing company. [Online]. Available: https://gigas.com/en/cloud-datacenter.
  • 8. Int J Elec & Comp Eng ISSN: 2088-8708 ļ² Earlier stage for straggler detection and handling using combined CPU test ā€¦ (Anwar H. Katrawi) 4917 BIOGRAPHIES OF AUTHORS Anwar H. Katrawi received B. Sc. In computer science from al Mustansiriya university, Iraq and M.Sc. in the computer information system from Arab Academy for Management, Banking and Financial Sciences of Jordan. He is currently a PhD Candidate in the school of Computer Sciences (NAV6) at Universiti Sains Malaysia. His research interests include Big data, data warehouse, machine Learning and data analytics. Rosni Abdullah is a professor in parallel computing and one of the national pioneers in the said domain. She was appointed Dean of the School of Computer Sciences at Universiti Sains Malaysia (USM) in June 2004, after having served as its Deputy Dean (Research) since 1999. She is also the Head of the Parallel and Distributed Processing Research Group at the School since its inception in 1994. Mohammed F.R. Anbar received his bachelor of Computer System Engineering from Al-Azhar University, Palestine and M.Sc. in Information Technology from Universiti Utara Malaysia, Malaysia (UUM). He obtained his PhD.ā€Æin Advanced Internet Security and Monitoringā€Æfrom University Sains Malaysia (USM). He is currently a senior lecturer at National Advanced IPv6 Centre (NAv6), Universiti Sains Malaysia. Ammar Kamal Abasi received B. Sc. in computer information system from Jordan university of science and technology, and M.Sc. in the international business from the university of Jordan. He is currently a PhD Candidate in the school of Computer Sciences at Universiti Sains Malaysia. His research interests include evolutionary algorithms, nature-inspired computation, and their applications to optimization problems.