A Hybrid Method of CART and Artificial Neural Network for Short Term Load Forecasting in Power Systems

1. Hiroyuki Mori
Dept. of Electronics Engineering
Meiji University
Tama-ku, Kawasaki 214-8571
Japan
hmori@isc.meiji.ac.jp

2.  I. Objective
 II. Background
 III. Case Studies in CEPCO
◦ Proposed Method
◦ Simulation
 IV. Conclusion

3.  To construct an Intelligent Model for Short-
term Load Forecasting
 Input data (Rules, Knowledge, Feature
Extraction) Output Data
 Cause and Effect of Input and Output Data

4. Nonlinear
Time-Varian Para
Large-Scale
metered
Quasi Periodical Dynamical
Random-Like Discrete
Stochastic

5. Deregulation
Competition Power
networks
Distributed
Generation

6.  Expert Systems (1980~)
 Artificial Neural Net (1987~)
 Fuzzy Inference (1990~)
 Evolutionary Computation or Meta-Heuristics (1990~)
 Multi-Agent Systems (1995~)
 Data Mining (2000~)

7.  Expert Systems  Inference
 ANN  Learning
 Fuzzy  Classification
 Meta-heuristics  Optimization
 Distributed Systems
 Multi Agent
 Knowledge Discovery
 Data Mining

8.  To Understand Complicated Data with Some
Rules
 To Extract Important Features That are Known
and/or Unknown
 To Construct More Reasonable
Models/Strategies

9.  Load Forecasting
 Dynamic Security Assessment
 Power System Control Center
 Data Profiling of Customers, etc.

10. Large Data
Base
System Feature
Complexity
Operators Extraction
Uncertainty

11.  To Propose a Hybrid Method of Regression
Tree and ANN for Short-term Load
Forecasting in Electric Power Systems
 To Optimize the Structure of the Regression
Tree with TS Globally
 To Extract Some Simple Rules From Data Set,
i.e., Explain the Relationship between Input
and Output Data

12.  Japan Sea
 CEPCO: Chubu Electric Power Co.
 Osaka, Nagoya, Tokyo
 Pacific Beach

13.  Large Factories (Toyota Gr.)
 High Humidity

14.  Kalman Filtering (Toyoda, „70)
 Regression Model (Asbury, „75)
 ARIMA Model (Hagan, „77)
 Expert System (Rahman,‟88)
 ANN (El-Sharkawi, ‟91)
 Fuzzy Decision Model (Park, ‟91)
 Fuzzy Neural Net (Mori, „94)
 Simplified Fuzzy Inference (Mori, „96)
 Chaos Time Series Analysis (Mori, „96)
 DM-Based Approach (Mori, 2001)

15.  Regression Model
 ANN Model
 Neuro-Fuzzy Model
 Fuzzy Inference Model

16.  To enhance the Model Accuracy
◦ To Minimize the Maximum Errors
 To clarify the Relationship between Input and
Output Variables
◦ To Validate Their Own Rules
◦ To Find out New Rules

17.  To Play a Key Role in Power System Operation
and Planning
◦ To Smooth ELD and UC
◦ To Make Profit through Deregulated and
Competitive Power Markets
◦ (insert equation)

18.  Learning Data Fuzzy ANN y
 Learning Data Preprocessor Predictor y
 Prediction Model with Preprocessing
Technique

19. Cluster 1
Classifier
Learning Cluster 2
Data
Classifier Cluster 3

20.  Regression Tree as a DM Tools (To Find Out
Important Rules)
 Open Issue
 To Focus on Globally Optimal Classification
Rather Than Locally Optimal or Locally Quasi-
optimal One

21.  Data Mining
◦ To Discover Important Rules in Large Data Base
 Data Mining
◦ Pattern Recognition
◦ Fuzzy Theory
◦ Decision Tree, etc.
◦ (insert cahrt of Split and Root Node)

22.  Growth
◦ Minimization of Error after Splitting
◦ R(n)=V(n)/V0
◦ R(n):Error of Node n
◦ V(n): Variance of Learning Data Belonging to Node n
◦ V0: Variance of All Learning Data
 Pruning
◦ Simple Structure of Regression Tree
 Error Estimate
◦ Cross-Validation Method

23.  △R(s,t)=R(t)-R(tL)-R(tR)
 Where, R(s,t): Degree of Error Reduction in
Case where Attribute s at node t, s: Attribute,
t: Parent Node, R(t): Sum of Squared Error of
Parent Node, R(tL(r)): Error of Left-Side
(Right-Side) Child Node
 (Insert Chart)

24.  (insert equation)
Where, r: Error, rcv(*): Cross-Validation error,
Standard Deviation of Cross-Validation Error,
Pruned Tree Number

25. Decision Tree Output Conventional Methods
Classification Qualitative CART, ID3, C4.5
Regression Quantitative CART
Drawback of Regression Trees
-Classification Accuracy
(Locally Optimal Structure)

26. Methods Decision Tree Applications Model Structure
Wehenkel, „94 Classification Transient Local
[A1] Stability
Rovnyak, „94 Classification Transient Local
[A2] Stability
Proposed Regression Load Global
Forecasting
[A1] Wehenkel, et. Al., “Decision Tree Based Transient Stability Method
a Case Study,” IEEE Trans. on Power Systems, Vol. 9, No. 1, pp. 459-
469, Feb. 1994
[A2] Rovnyak, et. Al., “Decision Tree for Real-time Transient Stability
Prediction,” IEEE Trans. On Power Systems, Vol. 9, No. 3, pp. 1417-
1426, Aug. 1994.

27.  Definition
◦ Iterative Methods That Have Some Heuristics or
Simple Rules in Search Process
 Feature
◦ To Aim at Evaluating Highly Accurate Solutions
 Typical Meta-Heuristic Methods
◦ SA, GA & TS

28. Methods Analogies Parameters Solution CPU-Time Probability
Accuracy
SA Annealing -cooling Less Slower X
schedule
-
temperatur
e
GA Natural -population Less Slow X
Selection -crossover
-mutation
TS Adaptive -tabu More Fast
Memory length

29.  Adaptive Memory (Tabu List)
 Only One Parameter (Tabu Length)
 No Use of Random Numbers
 Transition Type Algorithm

30.  (insert image)
 (a) neighborhood search
◦ Red (Fixed Attribute): Blue (Free Attribute)  Tabu
List
 (b) Tabu List

31.  To Construct the Regression Tree with the
Globally Optimal Structure
 To Combine the Optimal Regression Tree with
MLP
 Optimal Regression Tree
◦ To Assign Input Variables to Split Nodes
◦ To Globally Optimize Combinations of Input
Variables with TS

32.  (insert image)
 (a) phase 1, (b) phase 2
 V(a), V(b), V(c): Input Variables Used as Split
Conditions
 Fig. 16 Constructing Process by CART

33.  (insert image)
 (a) phase 1, (b) phase 2
 Fig. 17 Constructing Process of Proposed
Regression Tree

34.  TS Solution: Splitting Attribute
 Cost Function: (insert equation)
 (insert graph)
 Fig. 18 Transfer of Splitting Attribute to TS
Solution

35.  Start Set Initial Conditions Generate New
Solutions (Combinations of Input Variables)
Evaluate Cross-Validation Errors of New
Solutions (Calculate Split Value?)
(Pruning) Select Best Solution
Terminated? Stop

36.  Target: One-Step- ahead Daily Maximum Load Forecasting
 Learning Data: Summer Weekdays in June to September „91-‟98
(Except „93 for Unusual Weather Conditions)
 Test Data: Summer Weekdays in June to September „99
 Size of Initial Tree: 31 Splitting Nodes
 Tabu Length: 12
 Conventional methods: CART-MLP and MLP
 Table 4 Parameters of MLP
Method Learning Moment Iteration Hidden
Rate um Term s Unit
Proposed 0.01 0.6 10000 5
Method
CART- 0.02 0.1 10000 5
MLP
MLP 0.9 0.5 30000 5

37. No. Input Variables
A Day of the Week d+1
B Predicted Max Temperature
C Predicted Min Temperature
D Predicted Average
Temperature
E Predicted Min Humidity
F Predicted Discomfort Index
G Max Load day d
H Dif between max load on
days d and d-1
I Dif between avg temp
J Avg of max load
k Avg of avg temp

38.  (Insert graphs)

39.  (insert decision tree)

40.  (Insert decision tree)

41.  (Insert decision tree)

42. N(t) Rule
4 T(AV,d+1)> 28.05 C
L(md)> 0.845
Note L(md): Max Load on Day d

43. Methods Tree (sec) MLP (sec)
Proposed 17760 2.7
CART-MLP 7 4.3
MLP 27.6
Computer: FUJITSU S-7/7000U Model 45
SPECint_rate 95:422 (296MHz)
SPECfp_rate 95:561 (296MHz)

44.  (insert graph)

45.  1. This paper has proposed a Hybrid Method of
the optimal regression tree and MLP for short-
term load forecasting
 2. Tabu Search is used to globally optimize the
model structure of the regression tree
 3. The simulation results have shown that the
proposed method is more effective than CART-
MLP in terms of the average and the maximum
errors
 4. The proposed method allows to clarify the
relationship between input and output variables
through the systematic rules

46.  H. Mori and N. Kosemura, “Optimal Regression Tree Based Rule
Discovery for Short-term Load Forecasting,” Proc. Of 2001 IEEE
PES Winter Meeting, Vol. 2, pp.421-426, Columbus, USA, Jan.
2001
 H. Mori, N. Kosemura, K. Ishiguro and T. kondo, “Short-term
Load Forecasting with Fuzzy Regression Tree in Power Systems,”
Proc. Of 2001 IEEE International Conference on Systems, Man &
Cybernetics, pp. 1948-1953, Tuscon, AZ, U.S.A, Oct. 2001
 H. Mori, N. Kosemura, T. Kondo and K. Numa, “Data Mining for
Short-term Load Forecasting,” Proc. Of 2002 IEEE PES Winter
Meeting, Vol. 1, pp.623-624, New York, NY, USA, Jan. 2002
 H. Mori and Y. Sakatani, “An Integrated Method of Fuzzy Data
Mining and Fuzzy Inference for Short-term load forecasting,”
Proc. Of ISAP (CD-ROM), Limnos, Greece, Aug. 2003
 H. Mori, Y. Sakatani, T. Fujino and K. Numa, “An Efficient Hybrid
Method of Regression Tree and Fuzzy Inference for Short-term
Load Forecasting in Electric Power Systems, “A..Lofti and M.J.
Garibaldi (Eds.), “Applications and Science in Soft Computing,”
pp.287-294, Springer, Berlin, Germany, Nov. 2003