Aurora MySQL source setup guide

This is a step-by-step guide on how to configure your Aurora MySQL instance for replicating its data via the MySQL ClickPipe.

info

We also recommend going through the MySQL FAQs here. The FAQs page is being actively updated.

Enable binary log retention

The binary log is a set of log files that contain information about data modifications made to an MySQL server instance, and binary log files are required for replication. Both of the below steps must be followed:

1. Enable binary logging via automated backup

The automated backups feature determines whether binary logging is turned on or off for MySQL. It can be set in the AWS console:

Setting backup retention to a reasonably long value depending on the replication use-case is advisable.

2. Binlog retention hours

The procedure below must be called to ensure availability of binary logs for replication:

If this configuration isn't set, Amazon RDS purges the binary logs as soon as possible, leading to gaps in the binary logs.

Configure binlog settings in the parameter group

The parameter group can be found when you click on your MySQL instance in the RDS Console, and then heading over to the Configurations tab.

Upon clicking on the parameter group link, you will be taken to the page for it. You will see an Edit button in the top-right.

The following settings need to be set as follows:

1. binlog_format to ROW.

2. binlog_row_metadata to FULL

3. binlog_row_image to FULL

Then click on Save Changes in the top-right. You may need to reboot your instance for the changes to take effect - a way of knowing this is if you see Pending reboot next to the parameter group link in the Configurations tab of the RDS instance.

tip

If you have a MySQL cluster, the above parameters would be found in a DB Cluster parameter group and not the DB instance group.

Enabling GTID Mode

Global Transaction Identifiers (GTIDs) are unique IDs assigned to each committed transaction in MySQL. They simplify binlog replication and make troubleshooting more straightforward.

If your MySQL instance is MySQL 5.7, 8.0 or 8.4, we recommend enabling GTID mode so that the MySQL ClickPipe can use GTID replication.

To enable GTID mode for your MySQL instance, follow the steps as follows:

1. In the RDS Console, click on your MySQL instance.
2. Click on the Configurations tab.
3. Click on the parameter group link.
4. Click on the Edit button in the top-right corner.
5. Set enforce_gtid_consistency to ON.
6. Set gtid-mode to ON.
7. Click on Save Changes in the top-right corner.
8. Reboot your instance for the changes to take effect.

info

The MySQL ClickPipe also supports replication without GTID mode. However, enabling GTID mode is recommended for better performance and easier troubleshooting.

Configure a database user

Connect to your Aurora MySQL instance as an admin user and execute the following commands:

1. Create a dedicated user for ClickPipes:

2. Grant schema permissions. The following example shows permissions for the mysql database. Repeat these commands for each database and host you want to replicate:

3. Grant replication permissions to the user:

Configure network access

IP-based access control

If you want to restrict traffic to your Aurora instance, please add the documented static NAT IPs to the Inbound rules of your Aurora security group as shown below:

Private access via AWS PrivateLink

To connect to your Aurora instance through a private network, you can use AWS PrivateLink. Follow our AWS PrivateLink setup guide for ClickPipes to set up the connection.