The Power of Virtual Network: Infrastructure as a Service Cloud Computing - W...axsh co., LTD.
Wakame-VDC is a very cutting edged fundamental software to realize IaaS Cloud Computing. The virtual networking capability already was implemented on this software. Please visit to https://github.com/axsh/wakame-vdc/rpmbuild to try it on your server!
The Power of Virtual Network: Infrastructure as a Service Cloud Computing - W...axsh co., LTD.
Wakame-VDC is a very cutting edged fundamental software to realize IaaS Cloud Computing. The virtual networking capability already was implemented on this software. Please visit to https://github.com/axsh/wakame-vdc/rpmbuild to try it on your server!
Here is a best practice of Carrier-Grade AWS security, once you've copied the AWS cloud formation script provided by NTT DCOOMO. Let's open a new vista with this presentation.
(拡散希望。この夢の部分が伝わらないと、道楽と思われる)どうしてこういう活動をするのか? スタートアップがいきなり電話会社と同レベルのセキュリテイとデータアナリティクスを手に入れる。サービス企業がシステム構築のイニシアティブが取れるのです。そうすれば、より企業連携がより緊密にできる。企業連携のイノベーションが新たに生まれる。標準化重視のIndustrie4.0より先に行けるのです。横方向の展開で新たなバリューチェーンがしっかりと生まれる。
Some might think Docker is for developers only, but this is not really the case.Docker is here to stay and we will only see more of it in the future.
In this session learn what Docker is and how it works.This session will be covering core areas such as volumes, but also stepping it up to a few tips and tricks to help you get the most out of your Docker environment.The session will dive into a few examples of how to create a database environment within just a few minutes - perfect for testing,development, and possibly even production systems.
Machine Learning explained with Examples
Everybody is talking about machine learning. What is it actually and how can I use it?
In this presentation we will see some examples of solving real life use cases using machine learning. We will define Tasks and see how that task can be addressed using machine learning.
SQL Server 2017でLinuxに対応し、その延長線でDocker対応やKubernetesによる可用性構成が組めるようになりました。そしてリリースを間近に控えたSQL Server 2019ではKubernetesを活用したBig Data Cluster機能の提供が予定されており、コンテナの活用範囲はさらに広がっています。
本セッションではこれからSQL Serverコンテナに触れていくための基礎知識と実際に触れてみるための手順やサンプルをお届けします。
【DLゼミ】XFeat: Accelerated Features for Lightweight Image Matchingharmonylab
公開URL:https://arxiv.org/pdf/2404.19174
出典:Guilherme Potje, Felipe Cadar, Andre Araujo, Renato Martins, Erickson R. ascimento: XFeat: Accelerated Features for Lightweight Image Matching, Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
概要:リソース効率に優れた特徴点マッチングのための軽量なアーキテクチャ「XFeat(Accelerated Features)」を提案します。手法は、局所的な特徴点の検出、抽出、マッチングのための畳み込みニューラルネットワークの基本的な設計を再検討します。特に、リソースが限られたデバイス向けに迅速かつ堅牢なアルゴリズムが必要とされるため、解像度を可能な限り高く保ちながら、ネットワークのチャネル数を制限します。さらに、スパース下でのマッチングを選択できる設計となっており、ナビゲーションやARなどのアプリケーションに適しています。XFeatは、高速かつ同等以上の精度を実現し、一般的なラップトップのCPU上でリアルタイムで動作します。
セル生産方式におけるロボットの活用には様々な問題があるが,その一つとして 3 体以上の物体の組み立てが挙げられる.一般に,複数物体を同時に組み立てる際は,対象の部品をそれぞれロボットアームまたは治具でそれぞれ独立に保持することで組み立てを遂行すると考えられる.ただし,この方法ではロボットアームや治具を部品数と同じ数だけ必要とし,部品数が多いほどコスト面や設置スペースの関係で無駄が多くなる.この課題に対して音𣷓らは組み立て対象物に働く接触力等の解析により,治具等で固定されていない対象物が組み立て作業中に運動しにくい状態となる条件を求めた.すなわち,環境中の非把持対象物のロバスト性を考慮して,組み立て作業条件を検討している.本研究ではこの方策に基づいて,複数物体の組み立て作業を単腕マニピュレータで実行することを目的とする.このとき,対象物のロバスト性を考慮することで,仮組状態の複数物体を同時に扱う手法を提案する.作業対象としてパイプジョイントの組み立てを挙げ,簡易な道具を用いることで単腕マニピュレータで複数物体を同時に把持できることを示す.さらに,作業成功率の向上のために RGB-D カメラを用いた物体の位置検出に基づくロボット制御及び動作計画を実装する.
This paper discusses assembly operations using a single manipulator and a parallel gripper to simultaneously
grasp multiple objects and hold the group of temporarily assembled objects. Multiple robots and jigs generally operate
assembly tasks by constraining the target objects mechanically or geometrically to prevent them from moving. It is
necessary to analyze the physical interaction between the objects for such constraints to achieve the tasks with a single
gripper. In this paper, we focus on assembling pipe joints as an example and discuss constraining the motion of the
objects. Our demonstration shows that a simple tool can facilitate holding multiple objects with a single gripper.
13. Using SMB Direct & Multichannel
Without Multichannel
•No automatic failover
•Can’t use full bandwidth
–Only one NIC engaged
–RDMA capability not used
With Multichannel
•Automatic NIC failover
•Combined NIC bandwidth available
–Multiple NICs engaged
–Multiple RDMA connections
Reference by Jose Barreto's Blog
13
21. SQL Server 2012 using
SMB3 file shares
By Mario Broodbakker, Insight
Technology Inc
21
22. whoami
• Joined Insight Technology in August 2011
mostly worked on SQL Server DMA.
Helped ITI acquiring Oracle DMA
techniques in 2003 already.
• Before that: over 10 years of performance
engineering work on Oracle and SQL
Server, working for Baan, Compaq and HP
in Holland and USA.
• Started DBA career 25 years ago..
22
23. Windows 2012 SMB
• New version of SMB (Server Message Block
protocol): SMB3.0
• New features that can be used for SQL
Server:
– SMB Direct: SMB using RDMA (see next slides)
– SMB Multichannel, use multiple NICs for
throughput and failover (demo, if time permits)
– VSS (volume snapshot service) for SMB file
shares, to enable backup/restore
– SMB transparent failover (not yet on IQube..)
– PowerShell scriptable
23
24. SMB Direct 1
• RDMA: Remote Direct Memory Access
– Low latency protocol
– Low CPU overhead
SQL
Server
user
SMB SMB
Client Server
NTFS kernel
RDMA RDMA
network network
RDMA RDMA
NIC NIC
24
25. SMB Direct 2
• Direct memory copy by NIC hardware (must
be RDMA capable)
– Bypass TCP/IP and OS stack: while NIC
hardware copies data, CPU can do other work
• Transport Offload
• Zero Copy
• Kernel Bypass
• Can be used over any RDMA capable NIC
like Infiniband, (10Gb) Ethernet.
• Different protocols: Infiniband, iWarp, RoCE
• SMB client decides when to use RDMA
25
26. Advantages SMB
• Ease of use: everybody ‘understands’ file
shares
• No configuration or specialist knowledge
necessary to manage
– Out-of-the-box (almost): SMB Multichannel,
automatic failover, auto detection
– DBA can use standard perfmon counters to
understand IO load
26
27. SQL Server on SMB
• Possible since SQL Server 2008R2
http://sqlcat.com/sqlcat/b/technicalnotes/archive/2010/05/07/top-10-hidden-gems-in-sql-2008-r2.aspx
– Create database and logfiles on shares:
Create Database tpch100g on
Primary (name=tpch100g,
filename='¥¥fs1¥disk1¥tpch100g.mdf',size=10mb,filegrowth=0),
FileGroup iqdemo ..etc..
– But needs Windows 2012 to use SMB Direct
– Make sure SQL Server (domain) account has
permissions on the file share, and folders.
– SQL Server 2012: System DB on file share: install
root directly on share
– SQL Server AlwaysOn Failover Cluster supported
http://www.sqlskills.com/blogs/jonathan/post/Failover-Clustering-without-a-SAN-SQL-Server-2012-and-SMB-for-Shared-Storage.aspx
27
28. Verify RDMA
• Why? SMB can choose to use RDMA or IP at
runtime
• Also not all inbox drivers (and firmware
combinations) work.
– Perfmon Counters: RDMA Activity, SMB Direct
Connection .(SMB Server/Client Shares)
– PowerShell: Get-NetAdapterRDMA, Get-
SmbServerNetworkInterface, Get-
SmbClientNetworkInterface
– netstat.exe -xan | ? {$_ -match “445”}
28
30. Mellanox toolbox
C:¥Program Files¥Mellanox¥MLNX_VPI¥IB¥Tools>ibstat
CA 'ibv_device0'
CA type:
Number of ports: 1
Firmware version: 0x2000903e8
Hardware version: 0xb0
Node GUID: 0x002590ffff2ed2bc
System image GUID: 0x002590ffff2ed2bf
Port 1:
State: Active
Physical state: LinkUp
Rate: 40
Real rate: 32.00 (QDR)
Base lid: 16
LMC: 0
SM lid: 15
Capability mask: 0x90580000
Port GUID: 0x002590ffff2ed2bd
30
31. Benchmarking your setup
• Verify if all components work as expected
– Use an IO generator tool like sqlio or iometer
– Understand that you are generating serious
amounts of IO.
Therefore use multiple processes and
multiple threads
– Start testing local IO, then move on to network
– Use big (>10GB) test files to get smooth
performance
– Get a TPC-H kit from tpc.org (and scripts from
full disclosure reports posted by vendors)
31
32. Things to think about..
• Extent allocation: defaults to 1 extent (64k)
per file (round robin) ‘proportional fill’
• Use –E to get 64 extent proportional fill
• Traceflags: -T834 (large pages), -T2301
(DSS optimizations
http://support.microsoft.com/kb/920093
• Minimal logging options for dataloading,
see also:
– Microsoft Data loading performance guide
http://msdn.microsoft.com/en-us/library/dd425070.aspx
32
33. Insight Qube test
disks2.5 GB/sec
SQL Server s
ib
4.2 GB/sec
Windows 2012 IB
Switch
6.5 GB/sec ib
IB card: 3.5 GB/sec
ib
PCI-E Gen2
x8 4 GB/sec
ib ib
33
34. Summary
• SMB, RDMA, Infiniband combination
scales very well, mostly limited by PCI bus
speeds
• Easy to understand and administer
• Know your system, benchmark your
components and system.. or
• Insight Qube can do it for you !
34