How to Automate Data Sync Between Systems

Make a list of every system that stores customer, product, or transaction data. For each system, list the specific fields it manages. Common systems include:

• CRM (HubSpot, Salesforce, Pipedrive): Contact name, email, phone, company, deal stage, lifecycle stage, custom properties
• Email platform (Mailchimp, ActiveCampaign, ConvertKit): Subscriber email, name, list membership, tags, engagement metrics
• Support desk (Zendesk, Intercom, Freshdesk): Contact email, name, company, ticket history, satisfaction scores
• Billing system (Stripe, QuickBooks, Xero): Customer email, payment method, subscription status, invoice history
• Custom database: Application-specific user records, preferences, usage data

You do not need to sync every field between every system. Identify the critical fields that must stay consistent: email address, name, phone number, company name, subscription status, and customer tier. These are the fields where discrepancies cause real business problems.

One-way sync pushes changes from a source system to one or more destination systems. The source is always authoritative. This is simpler to build and eliminates conflict resolution. Use one-way sync when you have a clear primary system for each data type (CRM is the master for contact data, billing system is the master for subscription data).

Bidirectional sync allows changes in either system to propagate to the other. If a rep updates a phone number in the CRM, it syncs to the support desk. If a support agent updates the phone number in the support desk, it syncs back to the CRM. Bidirectional sync requires conflict resolution logic for cases where both systems change the same field around the same time.

Real-time sync uses webhook triggers: when a record changes, the system sends a webhook that starts the sync workflow immediately. Real-time sync is best for customer-facing data where delays cause problems (email address changes, subscription status changes).

Batch sync uses a scheduled trigger: every 15 minutes, every hour, or daily, the workflow queries for all records that changed since the last sync and processes them together. Batch sync is better for high-volume, lower-urgency data (analytics metrics, engagement scores, aggregate counts) and keeps API usage predictable.

Create a field mapping that defines how data translates between systems. This is more than just matching column names, it includes format differences, value transformations, and handling of fields that exist in one system but not another.

Common mapping challenges:

• Name formats: CRM stores "First Name" and "Last Name" separately. Email platform stores "Full Name" as one field. Your sync workflow concatenates first and last for the email platform, and splits on the first space for the CRM.
• Phone formats: CRM stores +1-555-123-4567. Billing system expects 5551234567. Normalize to a standard format before writing.
• Status values: CRM uses "Active," "Churned," "Paused." Billing system uses "active," "canceled," "past_due." Map each value explicitly.
• Date formats: One system uses YYYY-MM-DD, another uses MM/DD/YYYY, another uses Unix timestamps. Convert before writing.
• Missing fields: The CRM has an "industry" field. The email platform does not. Either skip this field in the sync, or store it in a custom field or tag in the email platform.

Document every mapping explicitly. When the sync breaks because a field format changed, the mapping document tells you exactly where to look.

The workflow structure depends on whether you chose real-time or batch sync.

Real-time sync workflow:

1. Webhook trigger fires when a record changes in the source system
2. Extract the changed fields from the webhook payload
3. Apply field transformations (format conversion, value mapping)
4. Look up the matching record in the destination system by unique identifier (email address or external ID)
5. If the record exists, update the changed fields. If it does not exist, create a new record.
6. Log the sync action (source record ID, destination record ID, fields synced, timestamp)

Batch sync workflow:

1. Scheduled trigger fires (every 15 minutes, hourly, etc.)
2. Query the source system's API for records modified since the last sync timestamp
3. Loop through each changed record
4. For each record: transform fields, look up in destination, upsert (update or create), log the action
5. Update the "last sync timestamp" so the next batch only processes new changes

Use the API nodes to call each system's API. Most CRMs, email platforms, and billing systems have well-documented REST APIs for reading and writing records.

In bidirectional sync, both systems might change the same field between sync cycles. The CRM rep updates the phone number to 555-1111 and the support agent updates it to 555-2222. Which one wins?

Common conflict resolution strategies:

• Last write wins: The most recently updated value takes precedence. Compare timestamps from both systems and keep the newer one. Simple but can lose intentional changes if timing is close.
• Source priority: Designate one system as authoritative for each field. The CRM always wins for contact info. The billing system always wins for subscription status. This eliminates ambiguity.
• Alert on conflict: When a conflict is detected, do not overwrite either value. Instead, flag the record and alert a human to resolve it manually. This is the safest approach for critical data but requires human time.

For deduplication, always look up records by a unique identifier before creating new ones. Email address is the most common unique key for contact records. If the lookup returns an existing record, update it. If it returns nothing, create a new one. Never create a record without checking for duplicates first.