Think the Synchronization!

ari (masanari Motohasi)
Institute of Mobile PC Technology
Purpose
●

How many PC or PDA do you have?
–

Office, Home, Mobiles ... any environments...

–

We wish that I had access ...
replication system
●

high availability ( 高可用性 )
–

●

redundancy ( 冗長化 ) / ritualized

consistency / concurrency ( 一貫性 )
...
ultimately ubiquitous network
●

インターネットにどんなところからでもアクセス
–
–
–

●

いつでも、どこでも、どんな方法でも
分散されていて、もっとも正規化が進む
情報の同期は考えなくていいのでは?

...
Viewpoints
●

like network layers (vertical)
–

●

physical/hardware to applications/software

Transparency or not (horizo...
Application/Software

Directions
dump/load
database

web [page/service]
grid/(mobile) agent
telnet
[process/object] remote...
Applications / Software
Web pages and services
●

●

one of the most distributed and normalized
solution with the Internet
standard in the world
–...
Web pages – Just Try It!
●

say nothing, because everybody knows (^ ^;)
Web Services – Just Try It!
●

SOAP / XML on HTTP
–

Apache SOAP (Java), Microsoft SOAP toolkit, SOAP
module(Perl)

–

con...
Remote Console, Remote Display
●

Remote Console
–

●

telnet (mainly UNIX)

Remote Display / Desktop
–

VNC (Virtual Netw...
Remote Console, Just Try It
●

telnet and telnetd (Unix)

●

Services for Unix (Windows) 29,800yen
–

functions
●
●
●

–

...
Remote Display, Just Try It
●

VNC (Virtual Network Computing)
–
–

●

for developers and tests only
check the security

p...
Groupware / Mail
●

communication!
–

●

from mails to workflow systems

Internet Mail System (on SMTP/POP3/IMAP etc)
–
–
...
Internet Mail, Just Try It!
●

Everybody knows in many OS and devices
–
–

●

Windows, Macintosh, UNIX, PDA, Phones, JRE.....
Groupware, Just Try It
●

Microsoft Exchange
–

●

only Microsoft world( ?NET technology)

Louts Notes / Domino
–
–

●

cl...
Database Management System
●

to be one of most normalized
–

●

keep isolation by “transaction”

SQL is the standard in t...
Database, Just Try It
●

Sybase Adaptive Server Enterprise 11.0 – 12.5 *

●

Oracle8i *

●

Microsoft SQL Server *

●

My ...
replication – mirroring
●

write the many devices as same time
applications / users
read/write
controller

controller
WAN?...
replication – copy/ snapshot
●

write the transaction to the primary
–

copy / snapshot to the secondary (only one way)

r...
replication - data trigger
●

the trigger
–

one way

–

performance overhead

–

very complexly

–

rollback is not easy
...
replications – stored & forward

difference
dump transaction

queue, DB, file, transaction log
2 phase commit
●

2 phase commit protocol is not redundant
–

just distributed

are you ready?

yes

yes

NO
replications – bi-directions
site A

site B

site A

site B

read/write

read/write

read/write

read/write

!
contention
...
make the contentions
●

when two or more “persons” update one
information as same time.
–

objects(Beans), file, transacti...
contention – lock
●

blocking oriented
–

DBM, File Systems

–

use the wait queue

–

deadlock
lock

B (lock waits)

upda...
contention Optimistic Concurrency Control
●

restart oriented
–

CVS, H/A EJB

time
update

re-open

B

×

update

open

o...
Version Controls
●

not only “source” but “documents”

●

CVS [Concurrent Versions System]
–
–

shell (command prompt) and...
File sync / replications
●

rsync

●

Microsoft Products
–

Briefcase (Windows 95 -) *

–

Replication Service (Windows NT...
rsync, Just Try It!
●

rsync on NT (run as services)
–

●

easy?

rsync algorithm
–

low-bandwidth high-latency bi-directi...
Microsoft Briefcase, Just Try It
●

Briefcase
–

–

ブリーフケースは直接ケーブル接続またはリムーバブル ディ
スクを使用してコンピュータ間のファイル転送を頻繁に行う
場合に最適なツールです。ブ...
Microsoft Offline Files
●

●

オフラインファイルはネットワーク上の共有ファイルを操
作するのに最適なツールです。オフライン ファイルを
使用すると、ネットワークに接続していないときに変
更した共有ファイルを次にネッ...
Microsoft Directory Replication
Service
●

“Windows NT Core Technology” で知りましたが、
実際の運用で使用したことはありません。
–

使ったことがある方、どんな感じでしょ...
Microsoft ActiveSync, Just Try It
–

Desktop PC – Pocket PC (serial / infrared)

–

Outlook data (mails, schedules, tasks,...
ftptrek / mailtrek, Just Try It!
●

ftptrek (Java)
–

synchronize with the data
●
●
●

faster than Windows(SMB) copy
check...
Physical / Hardware
Remote Access
●

very basic technology (PPP, SLIP...)
–
–

●

dial-up to the node (ex. office or home)
via Internet access...
Remote Access (Internet)
●

same as dial-up, but NOT secure

●

SSH / VPN
–

need: the secure server on the Internet

–

n...
VPN, Just Do It
●

IPsec (commercial)
–
–

●

CISCO VPN 3005
connected to another site

IPsec (private)
–

under construct...
SoftEther Vitual Ethernet System
●

SoftEther Virtual Ethernet System
●
●
●

Ethernet on the HTTPS and SSH
Able to pass th...
mobile PC / PDA
●
●
●

皆さん、いろいろな使い方を教えてください!
notebook PC
PDA
Storages
●

SAN (Storage Area Networks)
–

fast/virtual Access (Fiber Channel)

–

snapshot make the backup in a second

●...
SAN, Just Try It
●

databases and file servers
–

●

critical missions

snapshots is great
–

fast access (fiber channel a...
NAS (at home), try it
●

HD-HLAN, Buffalo (Link Station)
–

change from the file server to HLAN (NAS)

–

low power, small...
Internet Hard Disk, Just Try It
●

accessible to the disk like a drive

●

test the “MYDISK” service of rimnet.
–

like th...
Removable Devices
●

HDD, Flash Memory(CF/SD)...
–

use as primary devices and backups

–

has the risks
●
●

●

if lost (...
RAID 1 or 5
●

HDD is one of the most fragile parts
–

redundancy the HDD
Backup, Just Do It
●

Tape
–
–

serial access

–
●

Just standard, but very expensive for personal user
DLT / DLT / LTO......
Enterprise Disaster Recover Site,
Just Do It
●

for Business Continuity Planning (BCP)
–

●

Reduce the risk! plug off?

r...
Thank you!
Upcoming SlideShare
Loading in …5
×

Think sync (July 20, 2003) 同期を考えよう

1,260 views

Published on

2003年に、携帯PC技術研究所で発表したThink syncの資料を発見した。
そこでは"同期"について、実際の経験に基づいて考えた。
同期についての考察をさらに発展させるためアップする。

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,260
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Think sync (July 20, 2003) 同期を考えよう

  1. 1. Think the Synchronization! ari (masanari Motohasi) Institute of Mobile PC Technology
  2. 2. Purpose ● How many PC or PDA do you have? – Office, Home, Mobiles ... any environments... – We wish that I had access one Information anywhere. ● – the complete networking However, there are the barriers by cost, security, out of range... ● Do you have backup routinely? ● We need to synchronize the information!
  3. 3. replication system ● high availability ( 高可用性 ) – ● redundancy ( 冗長化 ) / ritualized consistency / concurrency ( 一貫性 ) – contention ( 衝突回避 ) ● response time / reasonable ( 高性能 ) ● easy to use / manage ● warm or cold standby
  4. 4. ultimately ubiquitous network ● インターネットにどんなところからでもアクセス – – – ● いつでも、どこでも、どんな方法でも 分散されていて、もっとも正規化が進む 情報の同期は考えなくていいのでは? しかし、ネットワーク的につながっている必要がある – – – セキュリティとプライバシは大丈夫? 同じセグメントかファイアウォールを抜く必要 コスト、圏外、公衆電話?
  5. 5. Viewpoints ● like network layers (vertical) – ● physical/hardware to applications/software Transparency or not (horizontal). – normalized and distributed – “use caches” have the contended problem. – “make copies (backups)” have a lag time. – recovery time is very important ● ● clear the “contention problem”. hot or cold standby?
  6. 6. Application/Software Directions dump/load database web [page/service] grid/(mobile) agent telnet [process/object] remote display groupware / mail [communication] Version Controls[file] file sync. (rsync/ftpsync) backup [file] (Tape etc) [unit] Database [transaction] Transparency remote access inet (VPN / SSH) P2P dial up [any] mobile PC/PDA [any] SAN / NAS / SAN Removable Media Internet Disk (snapshot) HDD/CF/SD..[file] RAID 1|5 Physical/Hardware
  7. 7. Applications / Software
  8. 8. Web pages and services ● ● one of the most distributed and normalized solution with the Internet standard in the world – – ● “standard” HTML / XML / SOAP etc very reasonable with secure – SSL (server|client certification, encryption) – and the other – only good design (^_^;
  9. 9. Web pages – Just Try It! ● say nothing, because everybody knows (^ ^;)
  10. 10. Web Services – Just Try It! ● SOAP / XML on HTTP – Apache SOAP (Java), Microsoft SOAP toolkit, SOAP module(Perl) – connect to the other world. ● ● ● Internet Applications (100% pure Java) and legacy systems (Microsoft COM (ASP / VB / C)). “Alive Check” system for an services (SOAP). B2B applications (XML on HTTP POST) SOAP Microsoft COM J2EE SOAP (HTTP+VPN) B2B another sites
  11. 11. Remote Console, Remote Display ● Remote Console – ● telnet (mainly UNIX) Remote Display / Desktop – VNC (Virtual Network Computing) * ● – pcAnywhere * ● – Windows only X ● – Windows and Unix mainly UNIX Terminal Server (Windows)
  12. 12. Remote Console, Just Try It ● telnet and telnetd (Unix) ● Services for Unix (Windows) 29,800yen – functions ● ● ● – telnet daemon (password sync Windows to Unix) server/client/gateway for NFS server for NIS for developers ● all developers telnet, compile and test the applications
  13. 13. Remote Display, Just Try It ● VNC (Virtual Network Computing) – – ● for developers and tests only check the security pcAnywhere – control and administer to the servers on the branches ● ● for security patch and maintenance terminal service (Windows 2000) – another profiles
  14. 14. Groupware / Mail ● communication! – ● from mails to workflow systems Internet Mail System (on SMTP/POP3/IMAP etc) – – ● very simple protocol MIME Exchange / Notes – has replication options – share files, meeting agents, but depend on the special platform
  15. 15. Internet Mail, Just Try It! ● Everybody knows in many OS and devices – – ● Windows, Macintosh, UNIX, PDA, Phones, JRE... devices and nodes for the synchronized files – send the attached mail oneself. (MIME) – it's not useful (not seamless and automatically) ● – see “mailtrek” ... however the firewall problem is clear! (with encrypt) ● check the contents and words
  16. 16. Groupware, Just Try It ● Microsoft Exchange – ● only Microsoft world( ?NET technology) Louts Notes / Domino – – ● closed technology and culture (not open). the Intranet and workflow systems have rich functions – ● mail system, scheduler, address books, public files, databases, replications depend on one company ... risk!
  17. 17. Database Management System ● to be one of most normalized – ● keep isolation by “transaction” SQL is the standard in the world – Oracle, Sybase ASE, Postage SQL, MS SQL... ● but mainly use in the LAN. ● 2 phase commit ● replication server – transaction base.
  18. 18. Database, Just Try It ● Sybase Adaptive Server Enterprise 11.0 – 12.5 * ● Oracle8i * ● Microsoft SQL Server * ● My SQL ● PostgreSQL
  19. 19. replication – mirroring ● write the many devices as same time applications / users read/write controller controller WAN? databases / devices / repositories(?)
  20. 20. replication – copy/ snapshot ● write the transaction to the primary – copy / snapshot to the secondary (only one way) read / write copy dump snapshot
  21. 21. replication - data trigger ● the trigger – one way – performance overhead – very complexly – rollback is not easy read/write ! data updated write
  22. 22. replications – stored & forward difference dump transaction queue, DB, file, transaction log
  23. 23. 2 phase commit ● 2 phase commit protocol is not redundant – just distributed are you ready? yes yes NO
  24. 24. replications – bi-directions site A site B site A site B read/write read/write read/write read/write ! contention read only read only
  25. 25. make the contentions ● when two or more “persons” update one information as same time. – objects(Beans), file, transaction, page, law... time A B ? update open open update!
  26. 26. contention – lock ● blocking oriented – DBM, File Systems – use the wait queue – deadlock lock B (lock waits) update open update wait open A time B
  27. 27. contention Optimistic Concurrency Control ● restart oriented – CVS, H/A EJB time update re-open B × update open open A B
  28. 28. Version Controls ● not only “source” but “documents” ● CVS [Concurrent Versions System] – – shell (command prompt) and GUI – over HTTP, FTP, MAIL. – ported to any OS, but mainly in UNIX – ● optimistic concurrency control method controlling my documents. Microsoft Visual SourceSafe – for developers
  29. 29. File sync / replications ● rsync ● Microsoft Products – Briefcase (Windows 95 -) * – Replication Service (Windows NT) – Offline File Synchronization (Windows 2000 -) – Microsoft ActiveSync * ● ● Desktop PC – Pocket PC (serial / infrared) another products
  30. 30. rsync, Just Try It! ● rsync on NT (run as services) – ● easy? rsync algorithm – low-bandwidth high-latency bi-directional communications – great – maybe not secure, not use in the naked Internet...
  31. 31. Microsoft Briefcase, Just Try It ● Briefcase – – ブリーフケースは直接ケーブル接続またはリムーバブル ディ スクを使用してコンピュータ間のファイル転送を頻繁に行う 場合に最適なツールです。ブリーフケースを使用して、メイ ンコンピュータ上のファイルと他のコンピュータ上で変更した ファイルを同期させることができます。複数のブリーフケース を作成してファイルを整理することができます easy to use, but performance is no good (many files with Gateway2000 Handbook).
  32. 32. Microsoft Offline Files ● ● オフラインファイルはネットワーク上の共有ファイルを操 作するのに最適なツールです。オフライン ファイルを 使用すると、ネットワークに接続していないときに変 更した共有ファイルを次にネットワークに接続したとき に同期させることができます。 使ったことがある方、どんな感じでしょう?
  33. 33. Microsoft Directory Replication Service ● “Windows NT Core Technology” で知りましたが、 実際の運用で使用したことはありません。 – 使ったことがある方、どんな感じでしょう?
  34. 34. Microsoft ActiveSync, Just Try It – Desktop PC – Pocket PC (serial / infrared) – Outlook data (mails, schedules, tasks, addresses...) and files – easy to use ● but I am used to erased all data by miss operations.
  35. 35. ftptrek / mailtrek, Just Try It! ● ftptrek (Java) – synchronize with the data ● ● ● faster than Windows(SMB) copy check the size and hash of the file support the zipped, parallel and serial copies – – ● synchronize with database transaction distribute over the program file updated mailtrek (Java) – SOAP on SMTP/POP3/IMAP? HTTP on MIME? – Sorry, under construction
  36. 36. Physical / Hardware
  37. 37. Remote Access ● very basic technology (PPP, SLIP...) – – ● dial-up to the node (ex. office or home) via Internet access the any services on PPP – wired and seamless!
  38. 38. Remote Access (Internet) ● same as dial-up, but NOT secure ● SSH / VPN – need: the secure server on the Internet – need: Global IP, Static IP or Dynamic DNS – need: keep the security and the privacy ● ● – ex. Many site that has old version OpenSSH is attacked. ex. Do dial-up users need to open the port 23(telnet) for the Internet? SoftEther Virtual Ethernet System ● ex. Ethernet on the HTTPS and SSH
  39. 39. VPN, Just Do It ● IPsec (commercial) – – ● CISCO VPN 3005 connected to another site IPsec (private) – under construction – with FTTH (B-flets, NTT) – share the files, administrate the server
  40. 40. SoftEther Vitual Ethernet System ● SoftEther Virtual Ethernet System ● ● ● Ethernet on the HTTPS and SSH Able to pass through the firewall Take care for use.
  41. 41. mobile PC / PDA ● ● ● 皆さん、いろいろな使い方を教えてください! notebook PC PDA
  42. 42. Storages ● SAN (Storage Area Networks) – fast/virtual Access (Fiber Channel) – snapshot make the backup in a second ● Network Attached Storages ● Internet Disk – use the Virtual Disk on the Internet Servers
  43. 43. SAN, Just Try It ● databases and file servers – ● critical missions snapshots is great – fast access (fiber channel and huge cache) – the backups and recoveries are in a second. – but the recoveries need the time ● full backup a day, and “loading transactions” are needed.
  44. 44. NAS (at home), try it ● HD-HLAN, Buffalo (Link Station) – change from the file server to HLAN (NAS) – low power, small size and silent – schedule backups the file a day via “rsync” Godzilla file server RAID 5 20GB (4G×6) HLAN no redundancy 160GB
  45. 45. Internet Hard Disk, Just Try It ● accessible to the disk like a drive ● test the “MYDISK” service of rimnet. – like the disk drive on the Windows ● ● ● – no accessible through our firewall ● – web accessible 100MB, free show demo... need the authentication to proxy stable?
  46. 46. Removable Devices ● HDD, Flash Memory(CF/SD)... – use as primary devices and backups – has the risks ● ● ● if lost (stop the job?) if stolen (get out the information?) encryption – Encrypting File System (EFS) on windows 2000 ● – over NTFS 5.0 PGPdisk
  47. 47. RAID 1 or 5 ● HDD is one of the most fragile parts – redundancy the HDD
  48. 48. Backup, Just Do It ● Tape – – serial access – ● Just standard, but very expensive for personal user DLT / DLT / LTO... HDD – Random access – I use the bus-powered 2.5inch HDD with rsync (at home)
  49. 49. Enterprise Disaster Recover Site, Just Do It ● for Business Continuity Planning (BCP) – ● Reduce the risk! plug off? redundant the anything – – Wires, networks (include Internet) and servers – ● Database synchronized by ftptrek humans and business processes warm standby – if “the incident” is happened, switch in a minute.
  50. 50. Thank you!

×