Error and Flow Control Protocol
(EFCP) Design and
Implementation: A Data Transfer
Protocol for the Recursive
InterNetwork Architecture
Miquel	Tarzan*	
Eduard	Grasa	
Leonardo	Bergesio	
	
ed	forty	conferences	upon	people	who	had	not	made	me	any	harm”	
G.K.	Chesterton	on	his	trip	to	the	US
Outline
•  A	glimpse	at	EFCP	(and	thus	at	Delta-t)	
•  EFCP	ImplementaNon	
•  Data	structures	
•  Workflows		
•  Experiments	
•  Where	have	we	tried	this	data	transfer	protocol?	
•  CongesNon	control	in	FP7	PRISTINE	
•  RINA-based	IP	VPN	experiments	
•  Conclusions	and	quesNons
A glimpse at EFCP: Structure and Place
in RINA
•  Data	transfer	
•  Seq	numbers	
•  Ordering	
•  Data	transfer	control	
•  Flow	control	
•  Rtx	control	
•  Policies	for	CongesNon	control	
•  Explicit	CongesNon	NoNficaNon	Bit(s)	
•  State	Vector	
•  Keeps	both	funcNons	coordinated	
IPC	API	
Data	Transfer	 Data	Transfer	Control	 Layer	Management	
SDU	Delimi9ng	
Data	Transfer		
Relaying	and	
Mul9plexing	
SDU	Protec9on	
Retransmission	
Control	
Flow	Control	
RIB	
Daemon	
RIB	
CDAP	Parser/
Generator	
CACEP	
	
	
	
Enrollment	
Flow	Alloca9on	
Resource	Alloca9on	
Rou9ng	
Authen9ca9on	
State	Vector	
State	Vector	
State	Vector	
Data	Transfer		Data	Transfer		
Retransmission	
Control	
Retransmission	
Control	
Flow	Control	
Flow	Control	
Increasing	9mescale	(func9ons	performed	less	oIen)	and	complexity	
Namespace		
Management	
Security	Management
A glimpse at EFCP: Timers for Reliability
∆ 𝑡	parameter	for	inacNvity	Nmers	∆ 𝑡= 𝑀𝑃𝐿+ 𝐴+ 𝑅	
EFCP’s	Nmers	
MPL	
LifeNme	of	PDUs	
A	
Timer	for	ACKs	
R	
Timer	to	stop	RTXs	
3	∆ 𝑡		
Sndr	InacNvity	Timer	
2	∆ 𝑡		
Rcvr	InacNvity	Timer	
Bounded,	thus	
Reliability	Guaranteed
Implementation of EFCP: PCI Data
Structures
struct	pci	{	
	unsigned	char	*h;		
	size_t	len;	
};	
struct	du	{	
	struct	efcp_config	*cfg;	
	struct	pci	pci;	
	void	*sdup_head;		
	void	*sdup_tail;		
	struct	sk_buff	*skb;	
};
Implementation of EFCP: Its data
structures
EFCP	Container	
EFCP	instance	
DTP	Instance	
DTP	Policy	Set		
DTCP	Instance	
DTCP	Policy	Set		
State	Vector	Instance	CWQ	 SeqQ	
sndrLWE, rcvrRWE, ...
RTXQ	
A-Timer	 Rtx	Timer
mplementation of EFCP: PCI Data
Dst	
Addr	
Src	
Addr	
Dst	CEP-
Id	
Src		
CEP-Id	
QoS-Id	 Ctrl	Seq	
Num	
sndRWE	 rcvRWE	PDU		
type	
Dst	
Addr	
Src	
Addr	
Dst	CEP-
Id	
Src		
CEP-Id	
QoS-Id	
Seq	
Num	
DRF	 ECN	
PDU		
type	
Data	Transfer	PCI	(Data	PDU)	
Data	Transfer	Control	PCI	(Control	PDU)	
sndLWE	 …
Implementation of EFCP: Functions
•  It	has	to	work	in	DIFs	very	different	one	from	another,	thus	the	
mechanisms	must	be	decoupled	so	they	can	be	used	independently		
EFCP	instance	
DTP	
Sequence	numbers	
DTCP	
Flow	control		
Ordering	
No	duplicaNons	
Rtx	Control	
CongesNon	Control
Workflows: Design & APIctrl_device	(user	space)	
efcp_connec(on_create	
efcp_connec(on_destroy	
IPCP	or	KFA	
efcp_container_write(sk_buff)	 ipcp_enqueue(sk_buff,	port_id)	
RMT	Instance	
rmt_send(sk_buff)	
EFCP	Container	
EFCP	instance	
DTP	Instance	
DTP	Policy	Set		
DTCP	Instance	
DTCP	Policy	Set		
State	Vector	Instance	CWQ	 SeqQ	
sndrLWE, rcvrRWE, ...
RTXQ	
A-Timer	 Rtx	Timer	
efcp_receive(sk_buff)
EFCP Workflows: Loading the policy sets
:IraNStack	 plugin:LKM	 dtp:dtp_ps	
insmod	
mmod	
dtp_ps_publish	
select_policy_set	
create	
<<create>>	
transmission_control	
reconcile_conflict	
select_policy_set	
destroy	
<<destroy>>	
dtp_ps_unpublish
Workflows: Flow Allocation &
Synchronisation
•  EFCP	doesn’t	carry	out	flow	allocaNon	
•  Task	of	Flow	Allocator		
•  EFCP	takes	on	data	transfer	iniNalisaNon:	DRF
Experiments: Congestion Control
•  In	FP7	PRISTINE	policies	for	CC	were	invesNgated	
•  Applied	to	the	network	of	a	distributed	cloud	provider	
•  Custom	policy	sets	for	EFCP	and	RMT	
•  2	CC	soluNons	based	on	the	ECN	and	window-based	flow-control	
•  The	RMT	monitors	its	queues	and	if	they	grow	too	long	ECN	flag	is	raised	
•  EFCP	reduces	or	increases	the	credit	depending	on	ECN	marks
Experiments: RINA-based IP VPN
Green	customer	CPE	(IP)	
Blue	customer	CPE	(IP)	
Purple	customer	CPE	(IP)	
Orange	customer	CPE	(IP)	
Red	customer	CPE	(IP)	
Pink	customer	CPE	(IP)	
Provider	network		
(RINA-based)	
CR CR
CR
PE PE
PE
PE
PE
PEPE
PE
PEPE
CE
PE
CR
CE CE
CE
CE
CECE
CE
CE
CE CE CE
CE
CE
CE
CE CE CE CE CE CE
CE
CE
CE
PE
CRCR PEPE
CECE Ethernet
Ethernet
EthernetEthernet
Ethernet Backbone	DIF
IP	(Green	customer	IP	VPN)
Experimental	details	
•  Medium	scale	scen
•  40	physical	nodes	
•  Up	to	6	DIFs	
•  1000+	concurrent	fl
Remarkable	results:		
•  Performance	achieved	by
prototype	was	closed	to	
rate	1	Gbps	physical	NICs
mulNple	aggragated	flow
Conclusions
•  IT	WORKS!	
•  The	design	fulfills	the	requirements	laid	out	in	the	specificaNon	
•  The	implementaNon	is	reasonably	performant	
•  Not	the	work	of	fools

Error and Flow Control Protocol (EFCP) Design and Implementation: A Data Transfer Protocol for the Recursive Internet Architecture

  • 1.
    Error and FlowControl Protocol (EFCP) Design and Implementation: A Data Transfer Protocol for the Recursive InterNetwork Architecture Miquel Tarzan* Eduard Grasa Leonardo Bergesio ed forty conferences upon people who had not made me any harm” G.K. Chesterton on his trip to the US
  • 2.
    Outline •  A glimpse at EFCP (and thus at Delta-t) •  EFCP ImplementaNon • Data structures •  Workflows •  Experiments •  Where have we tried this data transfer protocol? •  CongesNon control in FP7 PRISTINE •  RINA-based IP VPN experiments •  Conclusions and quesNons
  • 3.
    A glimpse atEFCP: Structure and Place in RINA •  Data transfer •  Seq numbers •  Ordering •  Data transfer control •  Flow control •  Rtx control •  Policies for CongesNon control •  Explicit CongesNon NoNficaNon Bit(s) •  State Vector •  Keeps both funcNons coordinated IPC API Data Transfer Data Transfer Control Layer Management SDU Delimi9ng Data Transfer Relaying and Mul9plexing SDU Protec9on Retransmission Control Flow Control RIB Daemon RIB CDAP Parser/ Generator CACEP Enrollment Flow Alloca9on Resource Alloca9on Rou9ng Authen9ca9on State Vector State Vector State Vector Data Transfer Data Transfer Retransmission Control Retransmission Control Flow Control Flow Control Increasing 9mescale (func9ons performed less oIen) and complexity Namespace Management Security Management
  • 4.
    A glimpse atEFCP: Timers for Reliability ∆ 𝑡 parameter for inacNvity Nmers ∆ 𝑡= 𝑀𝑃𝐿+ 𝐴+ 𝑅 EFCP’s Nmers MPL LifeNme of PDUs A Timer for ACKs R Timer to stop RTXs 3 ∆ 𝑡 Sndr InacNvity Timer 2 ∆ 𝑡 Rcvr InacNvity Timer Bounded, thus Reliability Guaranteed
  • 5.
    Implementation of EFCP:PCI Data Structures struct pci { unsigned char *h; size_t len; }; struct du { struct efcp_config *cfg; struct pci pci; void *sdup_head; void *sdup_tail; struct sk_buff *skb; };
  • 6.
    Implementation of EFCP:Its data structures EFCP Container EFCP instance DTP Instance DTP Policy Set DTCP Instance DTCP Policy Set State Vector Instance CWQ SeqQ sndrLWE, rcvrRWE, ... RTXQ A-Timer Rtx Timer
  • 7.
    mplementation of EFCP:PCI Data Dst Addr Src Addr Dst CEP- Id Src CEP-Id QoS-Id Ctrl Seq Num sndRWE rcvRWE PDU type Dst Addr Src Addr Dst CEP- Id Src CEP-Id QoS-Id Seq Num DRF ECN PDU type Data Transfer PCI (Data PDU) Data Transfer Control PCI (Control PDU) sndLWE …
  • 8.
    Implementation of EFCP:Functions •  It has to work in DIFs very different one from another, thus the mechanisms must be decoupled so they can be used independently EFCP instance DTP Sequence numbers DTCP Flow control Ordering No duplicaNons Rtx Control CongesNon Control
  • 9.
    Workflows: Design &APIctrl_device (user space) efcp_connec(on_create efcp_connec(on_destroy IPCP or KFA efcp_container_write(sk_buff) ipcp_enqueue(sk_buff, port_id) RMT Instance rmt_send(sk_buff) EFCP Container EFCP instance DTP Instance DTP Policy Set DTCP Instance DTCP Policy Set State Vector Instance CWQ SeqQ sndrLWE, rcvrRWE, ... RTXQ A-Timer Rtx Timer efcp_receive(sk_buff)
  • 10.
    EFCP Workflows: Loadingthe policy sets :IraNStack plugin:LKM dtp:dtp_ps insmod mmod dtp_ps_publish select_policy_set create <<create>> transmission_control reconcile_conflict select_policy_set destroy <<destroy>> dtp_ps_unpublish
  • 11.
    Workflows: Flow Allocation& Synchronisation •  EFCP doesn’t carry out flow allocaNon •  Task of Flow Allocator •  EFCP takes on data transfer iniNalisaNon: DRF
  • 12.
    Experiments: Congestion Control • In FP7 PRISTINE policies for CC were invesNgated •  Applied to the network of a distributed cloud provider •  Custom policy sets for EFCP and RMT •  2 CC soluNons based on the ECN and window-based flow-control •  The RMT monitors its queues and if they grow too long ECN flag is raised •  EFCP reduces or increases the credit depending on ECN marks
  • 13.
    Experiments: RINA-based IPVPN Green customer CPE (IP) Blue customer CPE (IP) Purple customer CPE (IP) Orange customer CPE (IP) Red customer CPE (IP) Pink customer CPE (IP) Provider network (RINA-based) CR CR CR PE PE PE PE PE PEPE PE PEPE CE PE CR CE CE CE CE CECE CE CE CE CE CE CE CE CE CE CE CE CE CE CE CE CE CE PE CRCR PEPE CECE Ethernet Ethernet EthernetEthernet Ethernet Backbone DIF IP (Green customer IP VPN) Experimental details •  Medium scale scen •  40 physical nodes •  Up to 6 DIFs •  1000+ concurrent fl Remarkable results: •  Performance achieved by prototype was closed to rate 1 Gbps physical NICs mulNple aggragated flow
  • 14.