Final Year Project report

  • 366 views
Uploaded on

Application of Neural Network in Prediction

Application of Neural Network in Prediction

More in: Technology , Sports
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
366
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
16
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 1 List of Abbreviations ANN Artificial Neural Network GWL Ground Water Level RWH Rain Water Harvesting h1 Hidden layer 1 h2 Hidden layer 2 NARX Nonlinear Autoregressive Exogenous Model R Correlation Coefficient MSE Mean Square Error
  • 2. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 2 INTRODUCTION CHAPTER 1
  • 3. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 3 A prediction (Latin præ-, "before," and dicere, "to say") or forecast is a statement about the way things will happen in the future, often but not always based on experience or knowledge. While there is much overlap between prediction and forecast, a prediction may be a statement that some outcome is expected, while a forecast is more specific, and may cover a range of possible outcomes. Although guaranteed information about the future is in many cases impossible, prediction is necessary to allow plans to be made about possible developments; Howard H. Stevenson writes that prediction in business "... is at least two things: Important and hard." Football (Soccer) has becoming increasingly popular over the years. During the last decade, soccer’s biggest event, the FIFA World Cup, has attracted millions of fans worldwide; in fact, the viewership of the FIFA World Cup match arguably surpasses the Super Bowl event as soccer is the more widely played sport in the world. To showcase its international reach, the World Cup was hosted in Japan and South Korea in 2002. South Africa hosted its first World Cup in 2010, and Brazil will host the next World Cup in 2014. Even in the Middle East, soccer appeals to the audience in that part of the world, with Qatar hosting the World Cup in 2022. The exposure of soccer to different continents highlights the growing popularity for the sport that originated in Europe. With great viewership comes great opportunity for soccer viewers to bet on scores too. This is especially true in Asia, where soccer betting is common among soccer fans. For instance, in Singapore, the Singapore Pools, which is Singapore’s legalized gambling institution, allows Singapore citizens to bet on almost anything, with soccer betting being one of its largest revenue generators. These bets range from predicting the outcome of the score, the exact result of the score, the winning margin and whether a certain player will score first. They are based on analyses including whether a certain player is playing, the coach’s record against the opposition and the strategies that the coach has been using. These factors are all based on human analyses with a tinge of biasness in them, which is inevitable for any sport that involves human judgment. In this experiment it will be shown how neural networks and Neuroph Studio are used when it comes to problems of classification. Several architectures will be tried out, and it will be determined which ones represent a good solution to the problem, and which ones do not. 1.1 An Overview
  • 4. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 4 1.2 Football : The Game Football refers to a number of sports that involve, to varying degrees, kicking a ball with the foot to score a goal. The most popular of these sports worldwide is association football, more commonly known as just "football" or "soccer". Unqualified, the word football applies to whichever form of football is the most popular in the regional context in which the word appears, including association football, as well as football, Australian rules football, Canadian football, Gaelic football, rugby league, rugby union, and other related games. These variations of football are known as football codes. Various forms of football can be identified in history, often as popular peasant games. Contemporary codes of football can be traced back to the codification of these games at English public schools in the eighteenth and nineteenth centuries. The influence and power of the British Empire allowed these rules of football to spread to areas of British influence outside of the directly controlled Empire, though by the end of the nineteenth century, distinct regional codes were already developing: Gaelic Football, for example, deliberately incorporated the rules of local traditional football games in order to maintain their heritage. In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the twentieth century, several of the various kinds of football grew to become among the most popular team sports in the world. 1.3 Rules and facts of game There are 17 laws in total, each one briefly summarized below. Field of Play. The game can be played on either natural or artificial surfaces, the surface must be green and rectangular in shape. The two long sides of the rectangle are called touch lines and the two shorter sides are called goal lines. The field is divided in half by the halfway line. Ball. Must be spherical, made of leather (or similar) 68-70 cm in circumference and of a certain pressure.
  • 5. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 5 Number of Players. Two teams of no more than 11 players (one of which is the goalkeeper). A game cannot start if either team has less than 7 players. Players are divided into three combination : Defender , midfielder , Forward player . Equipment. Players must wear a jersey, shorts, stockings, shin guards and footwear. Referee. The referee ensures the Laws of the Game are respected and upheld. Assistant Referees. There may be at most 2 assistant referees. Duration of the Match. The game is played in 2 halves consisting of 45 minutes each. The half time interval must not exceed more than 15 minutes. At the discretion of the referee more time is allowed to compensate for any stoppage during play e.g. Due to substitutions or care and attention of injured players. Start and Restart of Play. A kick-off starts play at the start of the match or after a goal. A kick- off involves one player kicking the ball, from stationary, forward from the centre spot. All players must be in their own half prior to kick-off. A coin is tossed pre-game, the team which loses the toss are awarded the kick-off to start the game whilst the team that win the toss are allowed to choose which direction they want to play. After half time the teams switch direction and the other team will kick-off. After a goal is scored, the team which conceded the goal will kick-off to restart play. Ball in and Out of Play. The ball is out of play once a goal has been scored or when the referee has stopped the game. The ball is in play at all other times. Method of Scoring. The ball crosses the goal line inside the goal mouth. Offside. It is an offence for a player to be in contact with the ball when they are closer to the opponents' goal than both the ball and the second-last opponent. The offside rule exists to ensure there are always opponents (generally the goal keeper and a defender) between a player receiving
  • 6. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 6 the ball and the goal. Without the offside rule, play can become boring with repeated long balls being kicked to a player stood next to the goalkeeper for an easy goal. Fouls/Misconduct. These are many and varied, broadly speaking it is an offence to use excessive force whilst playing the game either deliberately or undeliberately or to handle the ball (unless you are a goal keeper). The referee may show the yellow card to caution players for less serious offences and the red card for more serious offences resulting in the player being sent off. Two yellow cards are equivalent to one red card. Free Kicks. Are given by the referee for fouls and misconduct. A free kick can either be direct or indirect. A goal can be scored directly from a direct free kick. A goal can only be scored from an indirect free kick if it touches at least one other player first. The free kick must be taken from a stationary position with that position varying depending on whether the free kick was given inside or outside the goal area and whether it's direct or indirect. The opposing team must be a minimum of 9.15 m from the ball when the free kick is taken. Penalty Kicks. Are given against a team when they commit an offence which would normally be awarded a direct free kick inside their goal area. The ball is kicked from stationary from the penalty spot. The opposing team must be outside of the penalty area and at least 9.15 m from the ball. Throw-in. Used to restart play after the whole of the ball has crossed the touch line. Goal kick. Used to restart play after a goal has been scored. Corner Kick. Is given when the whole of the ball crosses the goal line and was last touched by a member of the defending team (and no goal was scored). A corner kick is taken from inside the corner arc closest to the point where the ball crosses the goal line. The defending team must be at least 9.15 m from the ball when the corner kick is taken.
  • 7. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 7 1.4 Role of prediction in Football Predicting the outcome of sporting events is a business which has grown in popularity in recent years. Handicappers predict the outcome of games using a variety of mathematical formulas, simulation models or qualitative analysis. Early, well known sports bettors, such as Jimmy the Greek, were believed to have access to information that gave them an edge. Information ranged from personal issues, such as gambling or drinking to undisclosed injuries; anything that may affect the performance of a player on the field. Recent times have changed the way sports are predicted. Predictions now typically consist of two distinct approaches: Situational plays and statistical based models. Situational plays are much more difficult to measure because they usually involve the motivation of a team. Dan Gordon, noted handicapper, wrote “Without an emotional edge in a game in addition to value in a line, I won’t put my money on it”. These types of plays consist of: Betting on the home underdog, betting against Monday Night winners if they are a favorite next week, betting the underdog in “look ahead” games etc. As situational plays become more widely known they become less useful because they will impact the way the line is set. The widespread use of technology has brought with it more modern sports betting systems. These systems are typically algorithms and simulation models based on regression analysis. Jeff Sagarin, a sports statistician, has brought attention to sports by having the results of his models published in USA Today. He is currently paid as a consultant by the Dallas Mavericks for his advice on lineups and the use of his Winval system, which evaluates free agents. Brian Burke, a former Navy fighter pilot turned sports statistician, has published his results of using regression analysis to predict the outcome of NFL games. Ken Pomeroy is widely accepted as a leading authority on college basketball statistics. His website includes his College Basketball Ratings, a tempo based statistics system. Some statisticians have become very famous for having successful prediction systems. Dare wrote “the effective odds for sports betting and horse racing are a direct result of human decisions and can therefore potentially exhibit consistent error”. Unlike other games offered in a casino, prediction in sporting events can be both logical and consistent
  • 8. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 8 In short we can say that we can use prediction techniques to find out best combination for any team by predicting individual performance of players and the total effect for their performances in the team .Using this process we can select best combination for any playing condition and opponent easily and effectively. 1.5 Neural Network 1.5.1 Biological neural networks A neuron ( or nerve cell ) is a special biological cell that processes information (see Figure 1.8). It is composed of a cell body, or soma, and two types of out-reaching tree-like branches: the axon and the dendrites. The cell body has a nucleus that contains information about hereditary traits and plasma that holds the molecular equipment for producing material needed by the neuron. A neuron receives signals (impulses) from other neurons through its dendrites (receivers) and transmits signals generated by its cell body along the axon (transmitter), which eventually branches into strands and sub strands. At the terminals of these strands are the synapses. A synapse is an elementary structure and functional unit between two neurons (an axon strand of one neuron and a dendrite of another), When the impulse reaches the synapse’s terminal, certain chemicals called neurotransmitters are released. The neurotransmitters diffuse across the synaptic gap, to enhance or inhibit, depending on the type of the synapse, the receptor neuron's own tendency to emit electrical impulses. The synapse's effectiveness can be adjusted by the signals passing through it so that the synapses can learn from the activities in which they participate. This dependence on history acts as a memory, which is possibly responsible for human memory [3].
  • 9. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 9 Figure1.1 Biological Neuron [3] The cerebral cortex in humans is a large flat sheet of neurons about 2 to 3 millimeters thick with a surface area of about 2,200 cm2, about twice the area of a standard computer keyboard. The cerebral cortex contains about 10" neurons, which is approximately the number of stars in the Milky Way." Neurons are massively connected, much more complex and dense than telephone networks. Each neuron is connected to 103 to l04 other neurons. In total, the human brain contains approximately 1014 to l015 interconnections. Neurons communicate through a very short train of pulses, typically milliseconds in duration. The message is modulated on the pulse-transmission frequency. This frequency can vary from a few to several hundred hertz, which is a million times slower than the fastest switching speed in electronic circuits. However, complex perceptual decisions such as face recognition are typically made by humans within a few hundred milliseconds. These decisions are made by a network of neurons whose operational speed is only a few milliseconds. This implies that the computations cannot take more than about 100 serial stages. In other words, the brain runs parallel programs that are about 100 steps long for such perceptual tasks. This is known as the hundred step rule.12 The same timing considerations show that the amount of information sent from one neuron to another must be very small (a few bits). This implies that critical information is not transmitted directly, but captured and distributed in the interconnections-hence the name, connectionist model, used to describe A’s .
  • 10. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 10 1.5.2 Artificial Neural Network Artificial neural network is a biologically inspired mathematical tool used widely for prediction and classification. In basic Artificial neural network there are three layers of artificial neurons given below: 1) Input layer: The function of this layer is to take input. 2) Hidden layer: This layer process that input as per the given activation function. 3) Output layer: After processing by hidden layer output layer provides the output. Figure 1.2 Artificial neural network An Artificial neuron is a processing unit which process the input. The input is taken by
  • 11. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 11 a input vector x. Each edge connecting two neurons have some weight Wij shows the connection strength between two neurons. The weights are given in weight matrix W. 1.5.3 Models of a neuron A neuron is an information-processing unit that is fundamental to the operation of a neural network. We may identify three basic elements of the neuron model: Figure 1.3 Nonlinear model of a neuron 1. A set of synapses, each of which is characterized by a weight or strength of its own. Specifically, a signal by the synaptic weight at the input of synapse j connected to neuron k is multiplied. It is important to make a note of the manner in which the subscripts of the synaptic weight are written. The first subscript refers to the neuron in question and the second subscript refers to the input end of the synapse to which the weight refers. The weight is positive if the associated synapse is excitatory; it is negative if the synapse is inhibitory.
  • 12. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 12 2. An adder for summing the input signals, weighted by the respective synapses of the neuron. 3. An activation function for limiting the amplitude of the output of a neuron. The activation function is also referred to in the literature as a squashing function in that it squashes (limits) the permissible amplitude range of the output signal to some finite value. Typically, the normalized amplitude range of the output of a neuron is written as the closed unit interval [0, 1] or alternatively [-1, 1]. 4. The model of a neuron also includes an externally applied bias (threshold) wk0 = bk that has the effect of lowering or increasing the net input of the activation function. In mathematical terms, we may describe a neuron k by writing the following pair of equations: or in a matrix form   Types of activation functions [3] a) Threshold activation function (McCulloch–Pitts model)
  • 13. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 13 Figure 1.4 Threshold activation function. In this model, the output of a neuron takes on the value of 1 if the total internal activity level of that neuron is nonnegative and 0 otherwise. This statement describes the all- or-none property of the McCulloch–Pitts model. The McCulloch–Pitts model of a neuron is sometimes used with the output either -1 or +1: b) Piecewise-linear activation function
  • 14. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 14 Figure 1.5 Piecewise-linear activation function The amplification factor inside the linear region is assumed to be unity. The following two situations may be viewed as special forms of the piecewise linear function: 1. A linear combiner arises if the linear region of operation in maintained without running into saturation. 2. The piecewise-linear function reduces to a threshold function if the amplification factor of the linear region in made infinitely large.
  • 15. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 15 c) Sigmoid (logistic) activation function Figure 1.6 Sigmoid (logistic) activation functions. The sigmoid function is the most common form of activation function used in the construction of artificial neural networks. Whereas a threshold function assumes the value of 0 or 1, a sigmoid function assumes a continuous range of values form 0 and 1. Note also that the sigmoid function is differentiable, which is an important feature of neural network theory. The derivative of the sigmoid function has a nice property, which makes is easy to calculate:
  • 16. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 16 d) Hyperbolic tangent function Figure 1.7 Hyperbolic tangent activation function The hyperbolic tangent function can be easily expressed in terms of the logistic function: (2 × logistic function – 1). Its derivative is also easy to calculate: e) Softmax activation function One approach toward approximating probabilities is to choose the output neuron nonlinearity to be exponential rather than sigmoidal and for each pattern to normalize the outputs to sum to 1. Let c be the number of output neurons. Each output is generated by the following activation function: It is clear that the softmax activation function is a smoothed version of a winner- take-all nonlinearity in which the maximum output is transformed to 1, and all others reduced to 0. The use of softmax is appropriate when the network is to be used for estimating probabilities. For example, each output yi can represent a probability that the corresponding neural network
  • 17. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 17 input belongs to class Ci.
  • 18. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 18 1.5.4 Multilayer feedforward network Figure 1.8 A 10-4-2 fully connected feedforward network. The source nodes in the input layer of the network supply respective elements of the activation pattern (input vector), which constitute the input signals applied to the neurons (computation nodes) in the second layer (i.e. the first hidden layer). The output signals of the second layer are used as inputs to the third layer, and so on for the rest of the network. Typically, the neurons in each layer of the network have as their inputs the output signals of the preceding layer only. The set of output signals of the neurons in the output layer of the network constitutes the overall response of the network to the activation pattern supplied by the source nodes in the input layer. For brevity, the network shown above is referred to as a 10-4-2 network in that it has 10 source nodes, 4 hidden neurons, and 2 output neurons. As another example, a feed forward network with p source nodes, h1 neurons in the first hidden layer, h2 neurons in the second layer, and q neurons in the output layer is referred to as a p-hl-h2-q network [3]. A neural network is said to be fully connected if every node in each layer of the network is connected to every other node in the adjacent forward layer. If, however, some of the communication links (synaptic connections) are missing from the network, we say that the network is partially connected.
  • 19. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 19 1.6 Learning The ability to learn is a fundamental trait of intelligence. Although a precise definition of learning is difficult to formulate, a learning process in the ANN context can be viewed as the problem of updating network architecture and connection weights so that a network can efficiently perform a specific task. The network usually must learn the connection weights from available training patterns. Performance is improved over time by iteratively updating the weights in the network. ANNs' ability to automatically learn from examples makes them attractive and exciting. Instead of following a set of rules specified by human experts, ANNs appear to learn underlying rules (like input-output relationships) from the given collection of representative examples. This is one of the major advantages of neural networks over traditional expert systems. To understand or design a learning process, you must first have a model of the environment in which a neural network operates, that is, you must know what information is available to the network. We refer to this model as a learning paradigm. Second, you must understand how network weights are updated, that is, which learning rules govern the updating process. A learning algorithm refers to a procedure in which learning rules are used for adjusting the weights . There are three main learning paradigms: supervised, unsupervised, and hybrid. In supervised learning, or learning with a “teacher,” the network is provided with a correct answer (output) for every input pattern. Weights are determined to allow the network to produce answers as close as possible to the known correct answers. Reinforcement learning is a variant of supervised learning in which the network is provided with only a critique on the correctness of network outputs, not the correct answers themselves. In contrast, unsupervised learning, or learning without a teacher, does not require a correct answer associated with each input pattern in the training data set. It explores the underlying structure in the data, or correlations between patterns in the data, and organizes patterns into categories from these correlations. Hybrid learning combines supervised and unsupervised learning. Parts of the weights are usually determined through supervised learning, while the others are obtained through unsupervised learning. Learning theory must address three fundamental and practical issues associated with learning from samples: capacity, sample complexity, and computational complexity. Capacity concerns how many patterns can be stored, and what functions and decision boundaries a net-
  • 20. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 20 work can form. , Sample complexity determines the number of training patterns needed to train the network to guarantee a valid generalization. Too few patterns may cause “over- fitting” (where in the network performs well on the training data set, but poorly on independent test patterns drawn from the same distribution as the training patterns). Computational complexity refers to the time required for a learning algorithm to estimate a solution from training patterns. Many existing learning algorithms have high computational complexity. Designing efficient algorithms for neural network learning is a very active research topic. 1.6.1 Backpropogation learning Consider a multilayer feed forward network, such as the three-layer network of Figure. Figure 1.9 Three layer feedforward network The net input to unit i in layer k + 1 is (2.1) The output of unit i will be (2.2) For an M layer network the system equations in matrix form are given by (2.3)
  • 21. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 21 (2.4) The task of the network is to learn associations between a specified set of input-output pairs The performance index for the network is (2.5) Where is the output of the network when the qth input, is presented, and is the error for the qth input. For the standard backpropagation algorithm we use an approximate steepest descent rule. The performance index is approximated by (2.6) where the total sum of squares is replaced by the squared errors for a single input/output pair. The approximate steepest (gradient) descent algorithm is then (2.7) (2.8) where is the leaming rate. Define (2.9) as the sensitivity of the performance index to changes in the net input of unit i in layer k.
  • 22. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 22 Now it can be shown, using (2.l), (2.6), and (2.9), that (2.10) (2.11) It can also be shown that the sensitivities satisfy the following recurrence relation (2.12) Where (2.13) And (2.14) This recurrence relation is initialized at the final layer (2.15) The overall learning algorithm now proceeds as follows; first, propagate the input forward using (2.3)-(2.4); next, propagate the sensitivities back using (2.15) and (2.12); and finally, update the weights and offsets using (2.7), (2.8), (2.l0), and (2.11). 1.6.2 Marquardt-Levenberg modification While backpropagation is a steepest descent algorithm, the Marquardt-Levenberg algorithm is an approximation to Newton’s method. Suppose that we have a function V( ) which we want to minimize with respect to the parameter vector I, then Newton's method would
  • 23. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 23 be (2.16) Where is the Hessian matrix and is the gradient. If we assume that is a sum of squares function (2.17) then it can be shown that (2.18) (2.19) where J(x) is the Jacobian matrix (2.20) And (2.21) For the Gauss-Newton method it is assumed that and the update (3.16) becomes (2.22) The Marquardt-Levenberg modification to the Gauss-Newton method is (2.23) The parameter is multiplied by some factor ( ) whenever a step would result in an
  • 24. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 24 increased . When a step reduces , is divided by . Notice that when is large the algorithm becomes steepest descent (with step ), while for small the algorithm becomes Gauss-Newton. The Marquardt-Levenberg algorithm can be considered a trust- region modification to Gauss-Newton. The key step in this algorithm is the computation of the Jacobian matrix. For the neural network mapping problem the terms in the Jacobian matrix can be computed by a simple modification to the backpropagation algorithm. The performance index for the mapping problem is given by (3.5). It is easy to see that this is equivalent in form to (3.17), where and N = Q x SM. Standard backpropagation calculates terms like (2.24) For the elements of the Jacobian matrix that are needed for the Marquardt algorithm we need to calculate terms like (2.25) These terms can be calculated using the standard backpropagation algorithm with one modification at the final layer (2.26) Note that each column of the matrix in (3.26) is a sensitivity vector which must be backpropagated through the network to produce one row of the Jacobian.
  • 25. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 25 1.7 Project Objective Prediction is very useful thing for games .Here we are predicting the result of a football match. Artificial neural network provide the better prediction result comparing other techniques. Objective of this research work is given as:
  • 26. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 26 PROJECT LITERATURE CHAPTER 2
  • 27. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 27 2.1 Literature Survey A number of studies have been done on different forecasting. Various data mining techniques have been used in these studies for prediction, and the calculated results are too much close to the real values. Heesung Yoon et al. [8] done their study at coastal aquifer and made a comparison between artificial neural network and support vector machine for prediction. The results of the model performance show that root mean squared error (RMSE) values of ANN models are lower than those of SVM in model training and testing stages. However, the overall model performance criteria of the SVM are similar to or even better than those of the ANN in model prediction stage. Radha Krisna Balla [5] uses feed forward backpropogation artificial neural network for result prediction at English Premier League in 2006-07. The model provided the good fit and the predicted trend followed the observed data closely 65% correctness .This proves the artificial neural network can be used for football match result prediction. Monica Adya And Fred Collopy perform 48 studies , only eleven met all of criteria for effectiveness of validation and implementation. Of the remaining 38, 17 presented effective validations but suffered with respect to implementation. Eleven of these reported positive results despite implementation problems. Altogether then, of the 48 studies, 22 contributed to our knowledge regarding the applicability of NNs to forecasting and prediction. Nineteen (86%) of these produced results that were favour-able, three produced results that were not., NNs, when they are effectively implemented and validated, show potential for forecasting and prediction. Second, a significant portion of the NN research in forecasting and prediction lacks validity. Ahmad Abrishamchi et al. [7] investigate the capability of two ANN models to predict the urban groundwater level using different sets of available input data, and then we compare the results of these two models. A multi-input-single-output network has been trained using Levenberg-Marquardt algorithm. The aforementioned models are evaluated using three statistical performance criteria namely mean square error (MSE), root mean squared error (RMSE), and efficiency (R2). This paper identifies the proficiency of ANN modeling technique to capture the complex dynamics involved in urban groundwater level fluctuations. The results show the importance of input data selection and its effect on prediction accuracy.
  • 28. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 28 Sreenivasulu D and Deka P.C. [8] investigate the potential and applicability of Radial Basis Function (RBF) for forecasting groundwater level with limited data. The results showed that ANN with RBF network can be an effective tool for better groundwater level fluctuations forecasting with limited data. P. Sujatha1 and Dr. G.N. Pradeep Kumar [10] examined Performance of four types of functionally different artificial neural network (ANN) models, namely Feed forward neural network, Elman type recurrent neural network, Input delay neural network and Radial basis function network and fourteen types of algorithms, namely Batch gradient descent (traingd), Batch gradient descent with momentum (traingdm), Adaptive learning rate (traingda), Adaptive learning rate with momentum (traingdx), Resilient backpropagation (trainrp), Fletcher-Reeves update (traincgf), Polak-Ribiere update (traincgp), Powell-Beale restarts (traincgb), Scaled conjugate gradient (trainscg), BFGS algorithm (trainbfg), One step secant algorithm (trainoss), Levenberg-Marquardt (trainlm), Automated regularization (trainbr) and Random order incremental training (trainr) were examined in order to identify an efficient ANN architecture and algorithm that can simulate the water table fluctuations using a relatively short length of groundwater level records. Results showed that Feed forward neural network trained with training algorithm Levenberg-Marquardt is suitable for accurate prediction of groundwater levels. Ioannis N. Daliakopoulos et al. [11] investigate and compare seven different types of network architectures and training algorithms in terms of model prediction efficiency and accuracy. The different experiment results show that accurate predictions can be achieved with a standard feedforward neural network trained with the Levenberg-Marquardt algorithm providing the best results for up to 18 months forecast. From all these studies we can conclude that feedforward neural network with Levenberg- Marquardt training algorithm shown to be a promising tool for ground water level prediction even with limited amount of data.
  • 29. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 29 PROPOSED METHODOLOGY & SIMULATION CHAPTER 3
  • 30. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 30 3.1 Proposed Methodology Any result of football match mainly based on performance of participating teams. This performance should be equal or best in different roles and areas of the game. For best performance of any team their players should be fit and inform .For this purpose teams are managing the performance and fitness data of the players and they evaluate this data to select right person for right place in the team. Then they forecast the performance of the team using historic data of players in different matches. It was a old approach to use only evaluation of the individual performance of any player here we does not relate the effect of individual performance in the team performance. So the prediction of the match was very tricky because we were not used the overall effect of performance by a player of the team, a department of the team (e.g. defense, forward, midfielder, goalkeeper). It is very easy to maintain the data of performance but to relate the data in the context of performance is difficult in different tools. We can use curve fitting and other different statics tool but these tools are failed to maintain the best error free result. Many models are based on mathematical calculations with dynamic programming, mainly in these calculation they use some mathematical concept such as probability with dynamic way and they predict result dynamically. For example WASP (Winning and Score Prediction) is a technique which is used in cricket at New Zealand for prediction of score and winning percentage of any team during the match. In this model we include the performance rating of team in four main parts of team. In these parts we include defense, goalkeeping, midfielder and forward. we collect the historic data of performance of players and final result of match in season 2013 to 2014 in Barclays premier league in match by match manner we randomly choose 106 matches .Then all these data sets are implemented in training of neural network with performance of players in different sectors as input and possible result of match as output. Then we can provide some test set for known matches and if predicted result is same as real then model is ready to use.
  • 31. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 31 3.2 Designing ANN models Designing ANN models follows a number of systemic procedures. In general, there are five basics steps: (1) Collecting data, (2) Preprocessing data, (3) Building the network, (4) Train, and (5) test performance of model as shown in Figure. Collecting data Preprocessing data Building the network Train Test Performance Figure 3.1 Designing ANN model  Data Collection Collecting and preparing sample data is the first step in designing ANN models.as input. First we need data set. For this problem we choose result of Premier League season 2011-12 . Because of great number of matches we randomly sampled 106 results. Each result has 8 input and 3 output attributes. Input attributes are: 1. Home team goalkeeper rating 2. Home team defence rating 3. Home team midfield rating 4. Home team attack rating 5. Visitor team goalkeeper rating 6. Visitor team defiance rating 7. Visitor team midfield rating 8. Visitor team attack rating
  • 32. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 32 Output attributes are: 1. Home team wins 2. Draw 3. Visitor team wins  Data pre-processing After data collection, three data preprocessing procedures are conducted to train the ANNs more efficiently. These procedures are: a. Solve the problem of missing data, b. Normalize data c. Randomize data. The missing data are replaced by the average of neighboring values during the same year. Normalization procedure before presenting the input data to the network is generally a good practice, since mixing variables with large magnitudes and small magnitudes will confuse the learning algorithm on the importance of each variable and may force it to finally reject the variable with the smaller magnitude. All input attributes are have integer values that can be very distant from each other. For example, goalkeeper rating can have maximum value of 100, and attack rating can have maximum of 300. In that case attack rating will influence more on problem than goalkeeper. To prevent that we will normalize data set using Max-Min normalization formula. B = (A - min⁡(A)) / (max⁡(A) - min⁡(A)) * ( D - C ) + C Where B is the standardized value, and D and C determines the range in which we want our value to be. In this case, D = 1 and C = 0. Normalized values are saved in PremierLeagueResults.txt file because they will be used for training and testing neural network 
  • 33. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 33    Building the network At this stage, the designer specifies the number of hidden layers, neuron in each layer, transfer function, weight/bias learning function, and performance function. In this work multilayer feedforward network is used with Levenberg-Marquardt learning algorithm.   Training the network In order to neural network learn the problem we need traaining data set. The training data set consists of input signals assigned with corresponding target (desired output). The neural network is then trained using one of the supervised learning algorithms, which uses the data to adjust the network's weights and thresholds so as to minimize the error in its predictions on the training set. If the network is properly trained, it has then learned to model the (unknown) function that relates the input variables to the output variables, and can subsequently be used to make predictions where the output is not known. In general, if you use a neural network, you will not know the exact nature of the relationship between inputs and outputs – if you knew the relationship, you would model it directly. The other key feature of neural networks is that they learn input/output relationship through training. There are two types of training used in neural networks, with different types of networks using different types of training. These are supervised and unsupervised training, of which supervised is the most common. In supervised learning, the network user assembles a set of training data. The training data contains examples of inputs together with the corresponding outputs, and the network learns to infer the relationship between the two. In other words, supervised learning is used for classification. For an unsupervised learning rule, the training set consists of input training patterns only. Unsupervised learning, on other hand, is used for clustering. Training set can be created in two ways. You can either create training set by entering elements as input and desired output values of neurons in input and output label, or you can create training set by choosing an option load file. The first method of data entry is time consuming, and there is also a risk to make a mistake when entering data. Since we already have training set we will choose second way.
  • 34. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 34  Testing the network The next step is to test the performance of the developed model. At this stage unseen data are exposed to the model. 147 samples are given to the network for the testing and evaluate the performance. In order to evaluate the performance of the developed ANN model quantitatively and verify whether there is any underlying trend in performance of ANN models, statistical analysis involving the correlation coefficient (R), the Mean square error (MSE) were conducted. 3.3 Creating the ANN model In this model we used the Neuroph Studio which is Java Neural Network development environment on top of NetBeans Platform based on Neuroph framework. It has been licensed under the Common Development and Distribution License (CDDL) . we are using Neuroph Studio 201207171143 (version 7.5) and java version 1.7 ,Java HotSpot(TM) Server VM 11.0 . Complete procedure of the project is given below – a. After normalizing all data we can start with Neuroph Studio. First we will create new Neuroph project then a new project is created and it will appear in the 'Projects' window, in the top left corner of Neuroph studio . b. In order to neural network learn the problem we need training data set. The training data set consists of input signals assigned with corresponding target (desired output). The neural network is then trained using one of the supervised learning algorithms, which uses the data to adjust the network's weights and thresholds so as to minimize the error in its predictions on the training set. If the network is properly trained, it has then learned to model the (unknown) function that relates the input variables to the output variables, and can subsequently be used to make predictions where the output is not known. c. In general, if you use a neural network, you will not know the exact nature of the relationship between inputs and outputs – if you knew the relationship, you would model it directly. Training set can be created in two ways. You can either create training set by entering elements as input and desired output values of neurons in input and output label, or you can create training set by choosing an option load file. The first method of data entry is time consuming,
  • 35. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 35 d. and there is also a risk to make a mistake when entering data. Since we already have training set we will choose second way e. After the network is trained, we click 'Test', in order to see the total error, and all the individual errors with respect to our training sets. f. To predict the future value of any match we have to create a specific test set in specified manner (same number and format of input and output variables as in training data set ) Some pictures are given below that are showing the process Figure 3.2 Creation of neuroph project
  • 36. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 36 Figure 3.3 Creation of training set
  • 37. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 37 Figure 3.4 training set with data
  • 38. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 38 Figure 3.5 Learning parameters before training
  • 39. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 39 Figure 3.6 network error graph
  • 40. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 40 Figure 3.7 weights for different neurons for a training set
  • 41. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 41 Figure 3.8 weights histogram for different neurons
  • 42. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 42 3.4 Equivalent Java code for model package predictingtheresultoffootballmatch; import java.io.FileNotFoundException; import java.io.IOException; import java.util.Arrays; import org.neuroph.core.NeuralNetwork; import org.neuroph.core.learning.DataSet; import org.neuroph.core.learning.DataSetRow; import org.neuroph.nnet.MultiLayerPerceptron; import org.neuroph.nnet.learning.MomentumBackpropagation; import org.neuroph.util.TrainingSetImport; import org.neuroph.util.TransferFunctionType; public class PredictingTheResultofFootballMatch { /* @param args the command line arguments */ public static void main(String[] args) { String trainingSetFileName = "PremierLeagueResults.txt"; int inputsCount = 8; int outputsCount = 3; System.out.println("Running Sample"); System.out.println("Using training set " + trainingSetFileName); // create training set DataSet trainingSet = null; try { trainingSet = TrainingSetImport.importFromFile(trainingSetFileName, inputsCount, outputsCount, ","); } catch (FileNotFoundException ex) {
  • 43. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 43 System.out.println("File not found!"); } catch (IOException | NumberFormatException ex) { System.out.println("Error reading file or bad number format!"); } // create multi layer perceptron System.out.println("Creating neural network"); MultiLayerPerceptron neuralNet = new MultiLayerPerceptron(TransferFunctionType.SIGMOID, 8, 30, 3); // set learning parametars MomentumBackpropagation learningRule = (MomentumBackpropagation) neuralNet.getLearningRule(); learningRule.setLearningRate(0.3); learningRule.setMomentum(0.7); // learn the training set System.out.println("Training neural network..."); neuralNet.learn(trainingSet); System.out.println("Done!"); // test perceptron System.out.println("Testing trained neural network"); testPredictingTheResultofFootballMatch(neuralNet, trainingSet); } public static void testPredictingTheResultofFootballMatch(NeuralNetwork nnet, DataSet dset) { for (DataSetRow trainingElement : dset.getRows()) { nnet.setInput(trainingElement.getInput()); nnet.calculate(); double[] networkOutput = nnet.getOutput(); System.out.print("Input: " + Arrays.toString(trainingElement.getInput())); System.out.println(" Output: " + Arrays.toString(networkOutput));
  • 44. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 44 } } } RESULT AND INTERPOLATION CHAPTER 4
  • 45. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 45 This chapter presents the best achieved results for feedforward backpropogation network model. The network has 8 neurons in input layer, 2 to 25 neurons in hidden layer and 3 s neuron for the predicted match result output. There are 106 samples are given to the network, which divided by tool in 3 parts, training, validation and testing. After the completion of training the Performance graph is given as: Figure 4.1 network error graph for training set The results are very impressive for the mean square error, and same as for the regression coefficient as shown in the graphs
  • 46. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 46 Figure 4.2 testing result for a testset
  • 47. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 47 CONCLUTION AND FUTURE WORK CHAPTER 5
  • 48. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 48 5.1 Conclusion In this model we simply create a architecture that can predict the result of a football match (best result for Barclays premier League) . As we know that we can forecast and predict the future values using neural network techniques and historical data . So we used the same concept and created a simple model for forecasting of match result , to achieve this we collect historical data ( performance rating of players for specific match with result of match ) from genius website of premier league ( Barclays Premier League ) . After data collection we proceed the preprocessing of data in this data cleaning, data normalization etc takes place . Normalized data set was used to train the network algorithm , for this we create several network architectures and and perform training to each architecture then after training and testing of each network architecture we check the performance of each architecture . Performance mainly based on minimum error (MSE). After these steps we can say that which architecture gave us the best result, so we select that best architecture and perform prediction for any upcoming match in the league. We can use same architecture for other leagues and matches but the accuracy of the prediction may be different from specified league.
  • 49. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 49 Neural network is very healthy area for forecasting or prediction. Neural network contains several different algorithms and techniques to forecast the values in various fields. Different techniques are based on historical data and different algorithms of training can easily forecast the values. We know that each technique of network needs data in its manner ( i.e. some format) for perfect prediction and minimum error. In this model we use 8 input variables which are mainly based on performance of players of both teams but we know that the performance of player not only depends upon their rating, it also depends on the place (match venue) , fitness of player ,weather , strategy of opposition etc . As we know that we use historical data for these attributes so it is difficult to collect the data for above given attributes from historic data but if we start collection of data of these attributes in present then we can easily and effectively create a perfect forecasting or prediction model in future . It is possible to create a model that can give best result for any match and any team, location, situation with historical data of different attributes. We also know that for perfect result also based on training algorithm and network architecture so we can say that modification and creation of new algorithm and architecture can also change the result and can give best result . So we can perform research work on it in future for various fields where forecasting and prediction can be applied. 5.2 Future work
  • 50. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 50 REFERENCES E
  • 51. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 51 [1] Kahn, J. (2003) Neural Network Prediction of NFL Football Games, “World Wide Web electronic publication, homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf. [2] Official website of Barclays Premier league ,for dats collection ,Available: http://www.premierleague.com/en-gb/matchday/results.html?paramComp_100=true&view =.dateSeason, accessed on 5 April, 2014. [3] M. Hajek, “Neural Networks” 2005, Available: www.sciag.ukzn.ac.za/comp/notes/NeuralNetworks2005.pdf accessed on 4 May 2014 [4] Martin T. Hagan and Mohammad B. Menhaj “Training Feedforward Networks with the Marquardt Algorithm”, IEEE Transactions on Neural Networks, Vol. 5, No.6, November 1994, pp 989-993. [5] Alan McCabe, "An Artificially Intelligent Sports Tipper," in Proceedings : 15th Australian Joint Conference on Artificial Intelligence (2002) [6] A. P. Rotshtein, M. Posner and A. B. Rakityanskaya, “Football Predictions Based on a Fuzzy Model with Genetic and Neural Tuning," Cybernetics and Systems Analysis Journal (2005) [7] P. D. Sreekanth, N. Geethanjali, P. D. Sreedevi, Shakeel Ahmed, N. Ravi Kumar and P. D. Kamala Jayanthi “Forecasting groundwater level using artificial neural networks” , Current science, vol. 96, no. 7, 10 april 2009. [8] David, John A.; Pasteur, R. Drew; Ahmad,M. Saif; and Janning, Michael C. (2011) "NFL Prediction using Committees of Arti_cial Neural Networks," Journal ofQuantitative Analysis in Sports: Vol. 7: Iss.2, Article 9. [9] ThePredictionTracker, (2007), “FootballPredictions", World Wide Web electronic publication,www.thepredictiontracker.com [10] Howard Demuth, Mark Beale, “Neural Network Toolbox, For use with Matlab” , User’s Guide, Version 4, The MathWorks
  • 52. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 52 [11] V. Nourani, A. Hosseini Baghanam, F. Daneshvar Vousoughi, M.T. Alami “Classification of groundwater level data using som to develop ann-based forecasting model” , International Journal of Soft Computing and Engineering (IJSCE), ISSN: 2231-2307, Volume-2, Issue-1, March 2012. [12] Heesung Yoon, Seong-Chun Jun, Yunjung Hyun , Gwang-Ok Bae, Kang-Kun Lee, “A comparative study of artificial neural networks and support vector machines for predicting groundwater levels in a coastal aquifer”, Journal of Hydrology 396 (2011), pp 128–138. [13] Faridah Othman and Mahdi Naseri “Reservoir inflow forecasting using artificial neural network”, International Journal of the Physical Sciences Vol. 6(3), , 4 February, 2011, pp. 434-440. [14] Purna C. Nayak,Y.R. Satyaji Rao and K. P. Sudheer “Groundwater Level Forecasting in a Shallow Aquifer Using Artificial Neural Network Approach”, Water Resources Management (2006) 20: pp 77–90. [15] http://www.mathworks.in/matlabcentral/fileexchange/19997-neural-network-for-pattern- recognition-tutorial/content/NN_tutorial.m, accessed on 11 May 2013. [16] Chi Leung Patrick Hui. (11 April, 2011). “Artificial Neural Networks – Application”, ISBN 978-953-307-188-6, InTec.
  • 53. APPLICATION OF NEURAL NETWORK IN FORECASTING Department of Computer Science & Engg., FIET Bareilly 53