Processor IntercommunicationIn part 1b we will look at the interconnectionnetwork between processors. Using theseconnections, various communication patternscan be used to transport information fromone or more source processors, to one or moredestination processors.
Processor Topologies (1)There are several ways in which processorscan be interconnected. The most importantinclude • Bus • Ring • Star • Mesh • Tree • Wraparound mesh • Fully connected • Hypercube
Topology IssuesBefore looking at some of the major processortopologies we have to know what makes acertain topology well or ill suited to connectprocessors.There are two aspects of topologies thatshould be looked at. These are scalability andcost (communication and hardware).
TerminologyCommunication is one-to-one when oneprocessor sends a message to another one.In one-to-all or broadcast, one processor sendsa message to all other processors. In all-to-onecommunication, all processors send theirmessage to one processor. Other forms ofcommunication include gather and all-to-all.
Connection PropertiesThere are three elements that are used inBuilding the interconnection network. Theseare the wire, a relay node, and a processornode. The latter two elements can at a giventime send or receive only one message, nomatter how many wires enter or leave theelement.
Tree Topology (2)Hardware cost 2p – 2, when p power of 2One-to-one 2 · 2log pOne-to-all (2log p) · (1 + 2log p)All-to-all 2 · (2log p) · (1 + 2log p)Problems Top node becomes bottleneck This can be solved by adding more wires at the top (fat tree)
2D-Mesh Topology (2)Hardware cost 2·(√p)·(√p – 1)One-to-one 2(√p – 1)One-to-all 2(√p – 1)All-to-all 2·√p / 2Remarks Scalable both in network and communication cost. Can be found in many architectures.
Hypercube Topology (2)Hardware cost (p / 2) · 2log pOne-to-one 2 log pOne-to-all 2 log pAll-to-all 2 · 2log pRemarks The most elegant design, also when it comes down to routing algorithms. But difficult to build in hardware.
Communication IssuesThere are some things left to be said aboutcommunication :• In general the time required for data transmission is of the form startup-time + transfer speed * package size. So it is more efficient to sent one large package instead of many small packages (startup-time can be high, e.g. think about internet)• Asynchronous vs. Synchronous transfer and deadlocks.
Asynchronous vs. SynchronousThe major difference between asynchronousand synchronous communication is that thefirst methods sends a message and continues,while the second sends a message and waitsfor the receiver program to receive themessage.
Example asynchronous comm. Processor A Processor B Instruction A1 InstructionA1 Instruction B1 InstructionB1 Send to B Send to B Instruction B2 Instruction B2 Instruction A2 Instruction A2 Instruction B3 Instruction B3 Instruction A3 Instruction A3 Instruction B4 Instruction B4 Instruction A4 Instruction A4 Receive from A from A Instruction A5 Instruction A5 Instruction B5 Instruction B5
Asynchronous Comm.• Convenient because processors do not have to wait for each other.• However, we often need to know whether or not the destination processors has received the data, this often requires some checking code later in the program.• Need to know whether the OS supports reliable communication layers.• Receive instruction may or may not be blocking.
Example synchronous comm. Processor A Processor B Instruction A1 InstructionA1 Instruction B1 InstructionB1 Send to B Send to B Instruction B2 Instruction B2 Instruction A2 Instruction A2 Instruction B3 Instruction B3 Instruction A3 Instruction B4 Instruction B4 Instruction A4 Receive from A from A Instruction A5 Instruction B5 Instruction B5
Synchronous comm.• Both send and receive are blocking• Processors have to wait for each other. This reduces efficiency.• Implicitly offers a synchronisation point.• Easy to program because fewer unexpected situations can arise.• Problem: Deadlocks may occur.
Deadlocks (1)A deadlock is a situation where two or moreprocessors are waiting to for each otherinfinitely.
Deadlocks (2) Processor A Processor B Send to B Send to A Receive from B Receive from ANote: Only occurs with synchronous communication.
Deadlocks (3) Processor A Processor BSend to B Receive from AReceive from B Send to A