1. The document proposes a heterarchical approach using intelligent cyber-physical products to schedule jobs in a partially flexible manufacturing system. 2. Under the proposed approach, each cyber-physical product uses distributed decision-making and learning to determine the optimal sequence of required services from flexible resources to complete its production. 3. Experimental results on three case studies show the proposed approach using cyber-physical products provides better performance than other traditional scheduling methods in terms of key metrics like average waiting time and completion time.

1. A novel approache for Cyber-Physical Manufacturing Systems optimization:
A heterarchical architecture with ditributed learning effect
Bouazza Wassim¹¯² · Sallez Yves²
¹LIO, University of Oran 1, 1526 El Mnaouer, Oran Algeria - ²LAMIH, UVHC, F-59313, Valenciennes, France
CONTEXTE PARTIALLY FLEXIBLE JOB-SHOP SCHEDULING PROBLEM
Proposed approach
(1) All SPs are assumed to be available at time 0.
(2) All CPPs arrive dynamically from time 0.
(3) Each CPP is assumed to have a priority (or critical-ity) that is a priori fixed.
(4) Each CPP requests a set of services, one at a time.
(5) Each SP has an input queuing zone, which is as-sumed to be infinite.
(6) Each SP can process only one service at a time.
(7) Once a service begins on an SP, it cannot be inter-rupted.
(8) CPP inter-resource and inter-cell transportation times are not considered.
(9) The availabilities and characteristics of SPs are supposed to remain
unchanged.
ASSUMPTIONS
Scheduling
Constraints
Dynamic job arrivals
Family-dependent
setup time
Family-dependent
processing times
Across different
partially flexible cells
!
With IoT and cyber-physical technologies, factories are up-
grading to Industry 4.0.
High flexibility of modern production systems involves
more complex issues with regard to scheduling production
jobs.
The particular case of partial flexibility makes the schedul-
ing more difficult, complicates the search space, and in-
creases the computation time [1].
This work proposes to deal with Partially Flexible Job-shop
Scheduling Problem using a heterarchical approach based
on intelligent cyber-physical products (CPPs).
j
Physical
Manufactured product
PhysicalLevelLogicalLevel
Cyber-
physical
Product
j
Decisional part Physical Product Resources
Service
Provider
Cell#2
Stage#2
Chain of
services
Srv1j
Srv2j
…
SrvIj
CSf
Key symbol
D
D
D
D
D
D
- The “physical” level, composed of physical products and re-
sources (e.g. machines).
- The “logical” level, which contains the computational entities
associated with the resources and products, respectively, man-
aging interactions to support the manufacturing process
Post-Decisional Evalua�on
Context Analysis & Iden�fica�on
Reinforcing
Selec�ng SP
Cyber-Physical Product
Scheduling applica�on
SPSR choice
Process Controller
2
3
Manufacturing
Informa�on System
Applying DR
DR Selec�on
5
Stochastic
parameters
Assignment Module
Services Chains
Database
Experiences
Database
Sequencing Module
Scheduler
Stochastic
parameters
Q1 Table
Q2 Table
Context
Chosen SPSR
Selected SP
Jobs sequence
Chosen DR
Wai�ng for service comple�on
Current service
1
4
6
7
A
B
For each service
8
1. The CPP uses the Service Chain Database to load the ordered list of services
corresponding to its product family.
2. According to the chain of services,
the current service is selected
3. At the required cell, the CPP gathers informa�on
from its local environment (e.g. IDen�fier, priori�es,
arrival �mes, and families, queued jobs, processing
and setup �mes).
The contextualiza�on module examines and iden�-
fies the current context.
4. The scheduler module divides the decisional process
into two steps: (A) assignment and (B) sequencing.
6. Once the new scheduling order has been sent,
the CPP then waits for the service to be completed.
8. The CPP refers back to the chain of services:
if it is not empty, the Process Controller triggers
a new decisional cycle. Otherwise, the CPP is
completely manufactured
5. To apply the resul�ng job sequence, orders
are sent to SPs to update the queues.
- Full
- Partial
- Single Machine
Flexibility (FCi )
- Without
- Homogenous
- Heterogeneous
Homogenous -
Resource-dependent -
Family-dependent -
Operating Time (PTCi) Setup Time (STCi)
7. The CPP then evaluates the decisions made
previ-ously by calcula�ng a Reward Func�on.
The values in the Q1 and Q2 tables are then
updated to save this post-decisional evalua�on
in the Knowledge Data-base.
Algorithm : Performance Indicator Pt
Output: value of P at instant t
1: Initially P=0 and y0=0
2: for j=1 to j=J
3: for all services Srvij of CPPj
4: if Cij ≤ t
5: P+=wiWij
6: yt+=1
7: end if
8: end for
9: end for
10: P/=yt
11: return P
Conclusion >>
In the present work, we were interested in partially flexible problems
with family-dependent setup and processing times. These complex
problems were directly inspired by real cases met in the pharmaceutical
and food industries.
These encouraging results open up interesting prospects. First, con-
cerning machine learning, it would be interesting to add an offline
learning phase for faster convergence toward efficient behaviour. Fur-
ther work must also be carried out to develop more effective contextu-
alization by introducing a specific context for each decisional step.
Some dynamic events such as breakdowns or maintenance tasks and
some constraints such as transportation times must be handled in the
future as well.
Algorithm WAWT AWT Cmax ∑sj ∑sT ∑pT
Case
study #1
Q-Algorithm (Best) 5.02 0.5268 21004 4325 3465 31984
SJF+LQE 5,94 0.5607 21005 4348 5173 37033
FIFO+LQE 5,94 0.5607 21005 4348 5173 37033
HPF+SST 6.83 0.7469 21004 1167 2033 39117
Q- Algorithm (Average) 6.11 0.6328 21006.33 5628.56 4856.68 38557.46
Case
study #2
Q-Algo (Best) 4 0.5196 21016 2450 5118 41191
SJF+LQE 6.09 0.5972 21012 6953 14947 43648
HPF+SST 4.19 0.5193 21019 2451 5120 41191
FIFO+LQE 6.1 0.5994 21012 6963 14957 43748
Q- Algo (Average) 4.20 0.6240 21015.33 3899.11 5248.62 41258.97
Case
study #3
Q-Algorithm (Best) 5.91 0.5835 21003 2594 2289 37755
SJF+LQE 6.49 0.6028 21005 4657 5657 37696
HPF+SST 7.17 0.7758 21004 2259 2360 37851
FIFO+LQE 6.18 0.5811 21005 4528 5437 37577
Q- Algorithm (Average) 6.32 0.6981 21006.64 3651.68 2984.12 37125.52
RESULTS
SST 3288
SST 16979
SST 3180
SPT 20141
SPT 2245
SPT 2203
LQE 2209
LQE 7937
LQE 20181
SQ 3777
SQ 2254
SQ 3851
0 5000 10000 15000 20000 25000
Case #3
Case #2
Case #1
LIFO 2148
LIFO 2156
LIFO 2204
HPF 2171
HPF 22787
HPF 2187
SJF 2269
SJF 2236
SJF 22875
FIFO 22827
FIFO 2236
FIFO 2149
0 5000 10000 15000 20000 25000
Case #3
Case #2
Case #1
Machine Selection Rules distibution Dispatching Rules distibution
Performance indicators synthesis
I. Kacem, S. Hammadi, and P. Borne, ‘Ap-
proach by localization and multiobjec-
tive evolutionary optimization for flexi-
ble job-shop scheduling problems’, IEEE
Trans. Syst. Man Cybern. Part C (Applica-
tions Rev., vol. 32, no. 1, pp. 1–13, Feb.
2002.
Experimental Data
Number of CPPs: J=10500, j ∈ [1... 10500]
Number of services: I=4, i ∈ [1... 4]
Total number of services requested: 29415
Number of families: F=9, f ∈ [1...9]
Priority range: wj ∈ [1...20]
CPP arrival times: Aij ∈ [1… 20999]
CPP arrival rate: 1 CPP per 2 time units
Multi-Agent
Simulator
Processing and setup times for case study #1 Processing and setup times for case study #3
f 1 2 3 4
Cell1
SP1
P1 2 2 2 2
S2 1 1 1 1
SP2
P 2 2 2 2
S 1 1 1 1
SP3
P 2 2 2 2
S - - - -
Cell2
SP4
P 2 2 x 2
S - - x -
SP5
P 1 1 x 1
S - - x -
SP6
P 2 2 x 2
S - - x -
Cell3
SP7
P 2 2 2 x
S 2 2 2 x
SP8
P 3 3 3 3
S 2 2 2 2
SP9
P 2 2 2 x
S 3 3 - x
Cell4
SP10
P 2 x 2 x
S 1 x 1 x
SP11
P 1 x 3 x
S 1 x - x
SP12
P 2 x 2 x
S - x - x
1Processing Time 2Setup Time
f 1 2 3 4 5 6 7 8 9
Cell1
SP1
P1 5 5 5 5 5 5 5 5 5
S2 2 2 2 2 2 2 2 2 2
SP2
P 5 5 5 5 5 5 5 5 5
S 2 2 2 2 2 2 2 2 2
SP3
P 1 1 1 1 1 1 1 1 1
S - - - - - - - - -
Cell2
SP4
P 2 2 x 3 3 3 3 x x
S 2 2 x 2 2 2 2 x x
SP5
P 2 2 x 3 3 3 3 x x
S 4 2 x 2 3 3 4 x x
SP6
P 3 3 x 2 2 2 2 x x
S - - x 5 - 1 - x x
Cell3
SP7
P 2 2 2 x x x x x x
S 2 2 2 x x x x x x
SP8
P 3 3 3 3 x x x 2 x
S 2 2 2 2 x x x 2 x
SP9
P 2 2 2 x x x x x x
S 3 3 - x x x x x x
Cell4
SP10
P 2 x 2 x 5 6 x x 9
S 6 x 2 x - - x x -
SP11
P 1 x 3 x 5 6 x x 9
S 2 x - x 2 2 x x 2
SP12
P 2 x 2 x 2 1 x x x
S - x - x 6 6 x x x
1Processing Time 2Setup Time
f 1 2 3 4 5 6 7 8 9Cell1
SP1
P1 2 2 2 2 2 2 2 2 2
S2 1 1 1 1 1 1 1 1 1
SP2
P 2 2 2 2 2 2 2 2 2
S - - - - - - - - -
Cell2
SP3
P 2 2 x 2 2 2 2 x x
S - - x - - - - x x
SP4
P 2 2 x 2 2 2 2 x x
S - - x - - - - x x
Cell3
SP5
P 3 3 3 3 x x x 2 x
S 2 2 2 2 x x x 2 x
SP6
P 2 2 x x x x x x x
S 3 3 x x x x x x x
Cell4
SP7
P 1 x 3 x 5 6 x x 1
S 1 x - x 1 1 x x 2
SP8
P 2 x 2 x 2 1 x x x
S - x - x 1 1 x x x
1Processing Time 2Setup Time
GUI of the simulation tool developped
Processing and setup times for case study #2