• Like
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
211
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. guarding your data Andrew Gilmartin senior developer agilmartin@crossref.org
  • 2. don’t loose the data •  Oracle holds the data in Boston Oracle × 2 + tape + off-site tape •  Oracle holds the data in Denver Oracle × 2 •  Denver less than 20 minutes out of sync with Boston
  • 3. syncing Boston & Denver •  Oracle's Oracle Streams … •  Other's Tungsten Replicator … •  Our's
  • 4. the tool’s context •  use existing tooling •  use a small set of dependent parts •  use already monitored parts •  Oracle •  file systems •  networks •  …
  • 5. the tool’s necessities •  reliable •  recoverable •  chatty •  alarming
  • 6. Oracle’s archive-log 1  archive-log represent the changes to the database since the last archive-log 2  discover created archive-log via Boston Oracle’s tracing-log 3  transfer archive-log to Denver 4  register archive-log with Denver’s Oracle, 5  discover archive-log applied via Denver Oracle’s tracing-log
  • 7. a bash script •  script & configuration are short at ~200 & ~10 lines respectively •  same script & configuration in Boston & in Denver •  Boston based script relies on Denver based script and vise versa •  a known set of functions
  • 8. significant event markers •  events triggered by tracing-log files •  events are recorded & coordinated via marker files •  events are created, transferred, registered, & applied •  markers are distinctively named, empty files in Boston arch_1_640475_754538743.arc.created archive-log’s name! marker!
  • 9. were what happens •  Boston continuously watches for archive-log creations invokes “transfer & register” process in Denver •  Denver continuously watches for archive-log applications invokes “applied” process in Boston •  Boston periodically investigate process’s status •  Boston & Denver periodically clean up
  • 10. some implementation notes •  humans are are good at •  check command exit recovery status •  use the operating system •  retry commands at ... ssh keys ... directory as database •  use configuration files rather than command line options •  Use command timeouts lengthening intervals … 5s 30s 300s … •  use bash's -E option
  • 11. some resources •  spark.sh & spark.conf •  ask Chuck for our tool, ckosher@crossref.org •  Giuseppe Maxia’s “Script it! Make professional DBA tools out of nothing” https://www.percona.com/live/mysql-conference-2013/ sessions/script-it-make-professional-dba-tools-outnothing