Mydbops 9th Opensource Database Meetup Presentation DMS(Database Migration Service)

1. DMS
(Database migration service)
A complete overview and working with DMS
Praveen GR
Associate Database consultant

2. About myself :-
❏ Associate Database Consultant Mydbops
❏ Interest in MySQL and it related Technologies
❏ Deep Proficiency in AWS Infrastructure
❏ Active Learner

3. Agenda

4. ❏ About DMS.
❏ Working with DMS.
❏ Limitation.
❏ Issues faced on DMS.

5. About DMS

6. ❏ DMS is a service providing by AWS.
❏ Used to migrate data between homogeneous (Same DBMS) and
heterogeneous (Different DBMS - Oracle, Microsoft SQL Server, MySQL,
MariaDB, PostgreSQL, Db2 LUW, SAP, MongoDB, and Amazon Aurora).
❏ Works well with different structure between source and target -
homogeneous.
❏ Migration pre-checks.
❏ Validation of data post full load.
❏ With ongoing replication.

7. Data Flow

8. Source
Database
DMS Process
Target
Database

9. Working with DMS

10. ❏ Database migration task.
❏ Replication instance.
❏ Endpoints.

11. Replication instance

12. Source
Database
Replication
instance
Target
Database

13. ❏ Intermediate instance used for migration.
❏ Load data using CSV.
❏ Same replication instance can be used for multiple migration task.
❏ CDC (Change data capture) is available for live replication.

14. ❏ Memory and CPU is needs to be taken care.
❏ Disk usage - Error log and CSV.
❏ Instance type for replication -> db.m6g.4xlarge.
❏ Use the latest version of DMS, for more stability and bug fixes.

15. Endpoints

16. Endpoint
Target
Source
User
credential
for server
access
User
credential
for server
access
Engine Engine

19. ❏ In the migration task, we need to mention the endpoint identifier.
❏ While creating endpoint identifier, Need to provide endpoint, user credential,
port for making the connection from the replication instance to source and
target.
❏ We can make the encrypted connection between source and target via
replication instance.
❏ We can’t change the endpoint, once it is mention in the migration task. But
we can edit the endpoint.

20. Endpoint testing
❏ Both the replication instance and target should in same VPC. Source may
be on-prem or cloud.
Replication
instance
Source
endpoint
Target
endpoint
Replication
instance

21. Migration task

22. DMS task
Source
endpoint
Target
endpoint
Replication
instance
Migration
type
Target
preparation
mode
Parallel
table
process
Filtration
Premigartion
assessments

23. ❏ Source and target endpoint.
❏ Replication instance.
❏ Migration type
❏ Migrate existing data.
❏ Migrate existing data and replicate ongoing changes.
❏ Replicate data changes only.

24. ❏ Target table preparation mode
❏ Do nothing
❏ Drop tables on target
❏ Truncate
❏ Table mappings
❏ Selection rules (Table filtering).
❏ Include/exclude tables.

25. ❏ Pre-migration assessment.
❏ Source table with LOBs but without primary keys or unique constraints.
❏ Unsupported source primary key types - composite primary keys
❏ Unsupported data types
❏ Need bucket with IAM role of read and write.

26. Limitation

27. ❏ Not all data types are supported for migration.
❏ You can't use AWS DMS to migrate from an on-premises database to another
on-premises database.
❏ For major limitation ->
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Limits.html

28. Issues faced in DMS

29. ❏ Single task will be the slower.
❏ It won’t migrate secondary index.
❏ Endpoint testing won’t check the privileges of the user.
❏ Reads IOPS due to validation.
❏ Not accurate percentage indication.

30. Workarounds

31. ❏ Split into multiple tasks.
❏ Structure has been restored to target before starting the migration process.
❏ Disabled the validation process during the migration task.
❏ Manual validation.

32. Queries ?

33. Reach Us @
info@mydbops.com