(Slides) Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments
•Download as PPT, PDF•
0 likes•683 views
Tamai, M., Shibata, N., Yasumoto, K. and Ito, M.: Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments, Proceedings of IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2005), Vol.1, pp.534-541, DOI:10.1109/CCGRID.2005.1558600 (May 2005) http://ito-lab.naist.jp/themes/pdffiles/ccgrid05-dbrokers.pdf In order to allow every user to extract aggregated computational power from idle PCs in the Internet, we propose a distributed architecture to achieve a market based resource sharing among users. The advantages of our proposed architecture are the following: (i) aggregated resources can be bought by one order, (ii) resource prices are decided based on market principles, and (iii) the load is balanced among multiple server nodes to make the architecture scalable w.r.t. the number of users. Through simulations, we have confirmed that the proposed method can mitigate the load at each server node to a great extent.
![Background ,[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
More from Naoki Shibata
More from Naoki Shibata (20)
Recently uploaded
Recently uploaded (20)
(Slides) Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments
- 1. Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments Morihiko Tamai, Naoki Shibata*, Keiichi Yasumoto, and Minoru Ito Nara Institute of Science and Technology, *Shiga University
- 10. Order Matching (cont.) Market broker Sellers buyers ((3, 3), 4) ((2, 1), 2) Buy price is less than the sell price
- 11. Order Matching (cont.) Market broker Sellers buyers ((3, 3), 4) ((2, 2), 3) ((2, 1), 2) ((1, 1), 5) Match From condition B, the buy-order matches the second sell-order, because the sell price is lower.
- 13. Logical Space for Orders CPU power 0 Memory amount Sell-order s 2 = ((80,40), 100) Sell-order s 1 = ((60,70), 90) max max Map each order to the point of the two-dimensional logical space s 1 (90) 60 70 s 2 (100) 80 40
- 14. Logical Space for Orders (cont.) Sell-order s 1 = ((60,70), 90) CPU power 0 max Memory amount max Sell-order s 2 = ((80,40), 100) s 2 (100) s 1 (90) Buy-order b Valid region of sell-order s 1
- 21. Order Replication (cont.) 0 max max m 2 m 1 m 3 m 5 m 4 m 6 m 8 m 7 m 9 s 1 (70) Problem: server nodes on the left below are overloaded to retain replicas. s 2 (80) s 3 s 4 s 5
- 34. Difference of matching results compared with centralized server node Our architecture achieves almost the same satisfaction degrees (>= 0.95) as the case of centralized server node Transmission delay among server nodes (ms) The case of centralized server node
- 37. Efficiency of one-to-many order matching Our method achieves higher matching probability than the simple method.
- 39. Thanks
- 48. (1000) (100)
Editor's Notes
- I’ll talk about our research, Distributed Market Broker Architecture for Resource Aggregation in Grid Computing Environments. This is a joint work with Naoki Shibata, Keiichi Yasumoto, and Minoru Ito.
- Recent progress of high speed networks and high performance PCs has make it possible to realize PC Grid and peer-to-peer computing systems such as seti at home, GIMPS, and cell computing. These projects have achieved more than super computer power at extremely low cost. In these projects, aggregated computing power is used for public objectives. But, we want to use these computing power for personal objectives, such as video encoding.
- To realize such a thing, following problems must be solved. That is, resource providers wouldn’t allow strangers to use their computers without appropriate incentive. In cell computing by NTT DATA, incentives for PC users are provided by giving them electronic currency. But, in the system, users can only provide resources. They can’t use other people’s resources.
- To address the problem, market-based resource sharing systems are proposed. In these systems, each user is resource seller or resource buyer. Resource seller receives e-money by offering his computational resource. And resource buyer pays e-money for using computational resource provided by seller. Here, the appropriate price of each resource is automatically decided based on supply and demand.
- When using the market mechanism, each seller wants to find buyer with highest price. Similarly, each buyer wants to find seller with lowest price. So, the existing systems utilize a market broker placed in a network node, which mediates between resource sellers and buyers.
- However, in the existing systems, market broker is implemented by centralized server, and with no detailed consideration of its scalability. Also, the systems can handle only about one-to-one order matching. But, if a user wants to use relatively large computation power, it is desirable to match a multiple set of sell-orders to one buy-order. To address the problems, we propose decentralized market broker architecture to increase scalability and treat one-to-many order matching in our architecture.
- This is the outline of my talk. Next, I will explain about the framework for market-based resource sharing.
- In our system, we suppose there are following types of resources. That is CPU power, memory amount, disk amount, and occupation time of resources. These amount of resources is represented by a resource vector r = (r1, r2, …, rn). Hereafter, for simplicity, we suppose there are only two types of resource. Each sell or buy order is represented by a tuple of resource vector and sell or buy price. For example, this sell-order ‘s’ means the seller wants to provide one thousand MIPS and eighty MB of resources at two hundreds of unit of currency.
- Here, I will explain how to match orders. In our system, order matching is conducted in the same way as the stock market at NYSE. That is, when the market broker receives a new order, it searches orders which satisfy condition A. The condition A means the seller provides enough resources than the buyer’s requirements. And, buyer provides enough e-money than the seller’s requirement. If there are multiple orders which satisfy condition A, one with the highest price is selected for sell-orders or one with the lowest price is selected for buy-orders.
- Here is the example. We suppose this seller sends a sell-order to market broker. Next, this buyer sends a buy-order. In this case, the two orders can’t match. Because the buy price is less than the sell price.
- Next, the second seller sends a sell-order, and for the same reason, the sell-order can’t match to the buy order. Next, the second buyer sends a buy-order. In this case, the buy-order can match to the both sell orders. From condition B, the buy-order matches the second sell-order, because the sell price is lower.
- Next, I will talk about distributed market broker architecture.
- In our method, we map each order to the point of the 2-dimensional logical space. In this space, x-axis and y-axis represent amount of CPU power and amount of memory, respectively. By using this space, for example, this sell-order s1 is mapped to this point. Here, this number is the sell price of the sell-order. Similarly, sell-order s2 is mapped to this point.
- Here, let pay attention to this red region. If new buy-order is registered in the red region, and its buy price is equal to or more than the sell price of s1, s1 can match to the buy order. So, we call this region as valid region of sell-order.
- In our method, we divide the whole space into sub-regions like this. Then, one server node is assigned to each sub-region.
- When a user registers a new order, he sends the order to the one of the server nodes. In this example, a new order S1 is sent to m8. After that, the order is forwarded to the appropriate server node using CAN routing. In this example, the resource vector of s1 is (80, 40), so it is forwarded to m6 like this. Finally, s1 is retained by m6.
- When a user registers a new order, he sends the order to the one of the server nodes. In this example, a new order S1 is sent to m8. After that, the order is forwarded to the appropriate server node using CAN routing. In this example, the resource vector of s1 is (80, 40), so it is forwarded to m6 like this. Finally, s1 is retained by m6.
- However, there is a problem. If a new buy-order b is registered to m8 like this, s1 and b are retained by different server nodes, respectively. So, s1 and b can’t match even though they satisfy matching condition.
- To cope with this issue, we make appropriate server nodes hold replicas of the order. In this example, replicas of s1 are retained by m4 to m9. But, when we send replication messages from m6 to these server nodes like this, the load of m6 will increase high.
- So, we use flooding based delivery method to send replication messages.
- In this approach, there is the other problem. For example, when these orders are registered to server nodes, all the replicas are retained by m7. So, server nodes on the left below are overloaded to retain replicas.
- To cope with this issue, we pay attention to sell price of sell-orders. In this example, the sell price of s1 is greater than the sell price of s2. So, when a new buy-order is registered to m7, it matches to s1. So, actual valid region of s2 is this sub-region. Thus, we can realize that m7 and m8 do not need to retain replicas of s2.
- Now, I will explain about one-to-many order matching method. Here, we suppose there are n sub-tasks. Let {b1, b2, …, bn} denote the set of buy-orders for the sub-tasks. We call each bi sub-buy-order. We suppose only the total budget is specified by user to buy resource for n sub-tasks. So, we register n sub-buy-orders as independent orders leaving their buy price values undefined.
- First, we think about a simple method. We suppose two sub-tasks {b1, b2}.
- First, we think about a simple method. We suppose two sub-tasks {b1, b2}.
- First, we think about a simple method. We suppose two sub-tasks {b1, b2}.
- Next, I will talk about experiments and its results.
- First, we conducted a experiment to evaluate what extent our distribute architecture can mitigate each server load. We used following settings. And, 15000 sell-orders and buy-orders are generated. The number of resource types is two. The amount to sell or buy each resource is determined by uniform random values in 0 to 100. And, sell or buy price is determined at random according to a Gaussian distribution whose mean and standard deviation are v1 + v2 and 30, respectively. Here, vi denotes the amount of i-th resource to sell or buy.
- In the experiment, we measured the messages received by each server node. The messages are categorized into the three types. It is buy and sell orders sent from client, replication messages for orders, and the deletion messages for replicas. We note that the second and third messages are overhead due to decentralization of market broker.
- This is the result of the experiment. In this figure, x-axis represents number of server nodes and y-axis represents average number of messages received per server node. This figure also shows the maximum and minimum numbers of messages received per server node of all server nodes. This figure shows that the number of messages received by each server node can be reduced significantly by increasing the number of server nodes. And we can see that the load is balanced among server nodes uniformly.
- Next, we conducted a experiment to evaluate the difference of matching results between our method and centralized server node. To evaluate this, we defined users’ satisfaction degree of matching result S as follows. Firstly, we define D. When a new order o1 matches an order o2 which is already retained by server node, D is defined as o1 buy price – o2 sell price, if o1 is buy-order. On the other hand, If o1 is sell-order, D is defined as o2 buy price – o1 sell price. If we use centralized server node for the market broker, order o2 which maximizes the value of D is always selected for order o1. On the other hand, if we use our method, the value of D is not always maximized since there are propagation delays among server nodes. So, we define satisfaction degree as DL / DG. Here, DL and DG are value of D when using decentralized servers and centralized server, respectively. In the experiment, we measured average values of S.
- This is the result. In this figure, x-axis represents time slot and y-axis represents average degree of users’ satisfaction. We measured average satisfaction by changing number of server nodes from 1 to 625. This line is the case of centralized server node. From this figure, we confirmed that our architecture achieves almost the same satisfaction degrees as the case of centralized server node.
- Next, we conducted a experiment to evaluate effectiveness of one-to-many order matching method. In the experiment, we measured probabilities of successful one-to-many matching using proposed method, and compared it to the case of simple method.
- Here, in the simple method, we generate n buy-orders using the settings of the first experiment, and registers these orders as independent orders. In this case, matching fails if there is a buy-order which is not matched to sell-order until the end of simulation time. In our method, we used the same buy orders as simple method. And, the total budget is the sum of buy prices of n buy-orders. We generated 15000 sell-orders, 13500 buy-orders, and 1500 buy-orders which include n buy-orders. Here, n is determined by exponential distribution whose average is 10.
- This is a result. In this figure, x-axis is number of server nodes, and y-axis is probability of successful one-to-many matching. From this figure, we can see our method achieves higher matching probability than the simple method.
- In this talk, we proposed distributed architecture for market broker. From experiments, we confirmed that our architecture can significantly reduce the number of messages processed by each server node, and greatly increases the availability of resources using one-to-many matching method. In the future work, dynamic division method of each sub-region assigned to a server node will be treated.
- In this approach, when a large number of orders are arrived at one server node in a short time period, the server node may be overloaded due to transmitting replication messages. So, we introduce a constant time slot T, and aggregate replication messages during every time interval T. After T, we deliver the aggregated messages to adjacent server nodes.
- Next, we describe about order deletion. There is a problem with the order deletion. For example, we suppose that buy-orders b1 and b2 are registered by m8 and m7, respectively. If these buy-orders are sent at almost the same time, s1 matches to both b1 and b2. To keep consistency, In our method, when a server node matches an order to a replica, let the server node check if the original order exists or not. If original of the order exists, the matching is confirmed. If not, the matching is canceled.
- When a server node receives sub-buy-order, it finds a sell-order with the lowest price which satisfies matching condition. So, these two sell-orders are selected. And, each server node sends the buyer tentative match message.
- When the buyer receives tentative match message from all server nodes which retain sub-buy-orders, it checks whether the total budget is enough to buy all sell-orders. If so, the buyer sends confirmation messages to these server nodes.
- When a server node receives confirmation message, it checks whether the sell-order still exists. If so, server node changes the state of sell-order to intermediate state, and sends ack message. Otherwise, server node sends nack message. Here, intermediate state means the order can’t match or tentative match to the other order.
- If the buyer receives ack messages from all server nodes, the sub-buy-orders finally matches to the sell-orders, and buyer sends commitment messages.
- Otherwise, if at lease one nack message received, buyer sends cancel messages to free intermediate state of sell-orders.