Skip to main content

Salesforce : DQE One - Unify Creating a Deduplication Process

Support DQE Software
  • Updated

Deduplication (called Duplicate in DQE One) processes are available with subscription credits

  • Before creating a deduplication process rule:
        • Optionally, map deduplication processes with custom and/or standard fields in Salesforce
              • use default mapping or create custom related mappings to use the “reparenting” functionality of DQE One (Creating a Deduplication mapping with custom and/or standard fields fields)
        • Create custom deduplication rules to your processes (see Creating a custom Deduplication rule)
        • Manage default deduplication rules (see Managing default Deduplication Rules)
        • Upload preexisting JSON Deduplication rules (see Uploading a preexisting JSON Deduplication Rule
  • When ready to create a deduplication process, or using default mappings (in Mappings tab) & default Deduplication rules (in Rules Manager tab), go to Creating a Deduplication process

 

 

Creating a Deduplication mapping with custom and/or standard objects  

 

Besides default mappings, you can choose to map DQE One deduplication processes to custom and/or standard objects of your choice in Salesforce:

 

  • In Mappings Tab > click on New

A screenshot of a computer Description automatically generated with medium confidence

 

  • In DQE_Mapping Name field, enter a free-form name for the mapping
  • In Description field, enter a free-form description of the mapping
  • In Process kind field, select the Duplicate check process
  • In Select a Data Source field, choose the Salesforce object for which you want to merge duplicates
  • In Select object to reparent when merging window, select each object you want to add to the parent record
  • Once you have selected an object, click on Jouer avec un remplissage uni icon
  • The object is selected and shown in the Selected window
  • Click on Save

The created mapping displays in Mappings Tab and will be available while you create your Process.

 

 

Managing default Deduplication Rules 

 

 

In Rules Manager Tab, default deduplication rules display for Matching, Parenting, and Merging

A screenshot of a computer Description automatically generated with medium confidence

To manage default rules:

  • In Sets…, select the object of your choice from the picklist
  • Select the rule of your choice
    • In the selected rule window, you can:
      • review different related rules per Matching/Parenting (Parent Records)/Merging (Merge)
      • rename a rule by clicking on the edit icon of its name
      • edit a rule by clicking on its edit icon
      • duplicate a rule by clicking on its duplicate icon
  • Back in Sets… window for your selected object, click on Save > Publish to save all edits in Matching/Parent Records/Merge

 

 

Creating a custom Deduplication Rule Set 

 

Besides default deduplication processes, you can choose to create a new deduplication Rule Set.

You can either duplicate an existing Rule Set or create a Rule Set from scratch.

 

Duplicating an existing Rule Set

Note : we recommend starting with Matching  

 

A screenshot of a computer Description automatically generated with medium confidence

In Rules Manager, in Sets… select a default set

Select the dropdown arrow to the right of the default set. 

Select Duplicate  

On the Matching tab, click on New Rule to add a rule to the existing Rule Set

Review the existing related field matches.  

      • Rule is activated - toggle to deactivate your rule. 
      • Duplicate current rule.
      • Delete current rule.
      • : existing Field Matches - to delete the selected field match (line)
      • : New Field Match create new field matches - to create a new field match (line)
      • edit Method Name, Method Description, etc.
      • duplicate an existing line. 

 

In Parent Records tab, click on New Rule to add a new rule to determine the parent record 

A screenshot of a computer Description automatically generated with medium confidence

  • : search for the field you want to use to determine the parent record rule.
  • : to delete the parent method (line)
  • edit the Method / Formula, Function Call, etc.

In the ‘Edit Parent Record Rule’ window, hold your cursor over the information icons to display functionality description.

 

On the Merge tab, the Salesforce fields with default merging rules are displayed.

A screenshot of a computer Description automatically generated with medium confidence

: search for the field you want to use with a merging rule  

: to modify the merging method (line) 

edit the Method / Formula, Change Function Call, etc .

 

In the ‘Edit Merge Rule’ window, hold your cursor over the information icons to display functionality description.

Note : you can use more than one merging method for each field except for “solo” methods. 

To publish the modified Rule Set click on Save > Publish.    

To export the file containing the default or customized rules, click on icon.

 

Creating a Rule Set from scratch  

Note: we recommend starting with Matching.   

A screenshot of a computer Description automatically generated with medium confidence

  1.  

 

In Rules Manager, in Sets… click on New.   

In Object > All Origins > search for and select the object on which you want to create a Rule Set.   

In Name: name your Rule Set.  

in Matching click on New Rule to create a rule (you can create as many as you need).  

In each rule you will create at least one field match.  

  1. Rule is activated - toggle to deactivate your rule 

  1. Duplicate current rule. 

  1. Delete current rule. 

  1.   : existing Field Matches - to delete the selected field match (line)  

  1. : New Field Match create new field matches - to create a new field match (line)  

  1. edit  Method Name, Method Description, etc.  

  1. duplicate an existing line.  

 

In Parent Records click on New Rule to create a new rule to determine the parent record .

  1. A screenshot of a computer Description automatically generated with medium confidence

: search for the field you want to use to determine the parent record rule. 

: to delete the parent method (line) 

edit the Method / Formula, Function Call, etc. 

In the ‘Edit Parent Record Rule’ window, hold your cursor over the information icons to display functionality description. 

 

In Merge the standard/custom Salesforce fields are displayed  :

A screenshot of a computer Description automatically generated with medium confidence

Note : to use the DQE’s results fields you have to run a process first to generate the corresponding fields.   

  

: search for the field you want to use with a merging rule  

: to modify the merging method (line)  

edit the Method / Formula, Change Function Call, etc 

 

In the ‘Edit Merge Rule’ window, hold your cursor over the information icons to display functionality description.  Note : you can use more than one merging method for one field except for “solo” methods.

 

To publish the modified existing set of rules, click on Save > Publish   

For exporting the file containing the default rule or your customized deduplication file, click on

icon.  

 

 

Uploading a preexisting JSON Deduplication Rule.

 

Caution: Note that uploading a preexisting JSON file will overwrite any existing rules.

A screenshot of a computer Description automatically generated with medium confidence

  • From the Installer tab > DQE Unify App Setup window, 
  • Click on Upload files button to load your JSON rule file or
  • Drag & drop your JSON rule file in Import Deduplication Rules window
  • Click on Save

 

Uploaded JSON rule file displays in Rules manager tab.

 

 

Create Deduplication Process

 

 

To create a process, you need to define its general settings, its source, and list view.

Audit options and Advanced options are also available if required. 

 

On the Processes tab, click on New. The Process Creation screen displays: 

A screenshot of a computer Description automatically generated

In the General Settings window:  

  • DQE-Process Name - enter a free-form name for the process. 

  • Data Quality Process - select 'Find Duplicates. 

  • Description - enter a free-form description of the process. 

 

In the Source window: 

  • In Select a Data Source field, select the object to which the process applies. Once a Data Source has been selected, Mappings and Active Steps sections will display
      • In Mapping field, select the mappings of your choice 

Note: this is an optional step if any previously custom mappings were created in Mappings (see Creating a Deduplication Mapping)

      • In Active Steps window, activate the rules of your choice for the selected object source

Note: Active Steps displays default rules and custom rules if any previously created from Rules manager (See Creating a custom Deduplication Rule). 

 

In the List View and Filters window: 

  • DQE One uses Salesforce List Views to determine which Salesforce records will be evaluated when looking for duplicates. If you would like to query a subset of your database rather than the entire Salesforce database, include filter criteria in yourlist view to limit the data being queried. For example, you may wish to only review records owned by a specific user or of a specific record type. 

 

  • DQE One also uses List Views to determine which fields will be displayed in your results.  You will want to include fields that assist you in determining what is a duplicate.

 

  • Salesforce List Views are limited to 15 fields so, you may want to remove specific fields in favor of fields of greater value in determining duplicates.

Select the preferred list view from the Existing List View section or

Manually Create List View 

        • In Add new, select Manually Created List View 
        • Recently Viewed screen related to the object displays 
        • Click on  the setup icon > select New in picklist 
        • In New List View window, enter a free-form name for the list view
            • The API name is automated in List API Name field

 

      • Select who can see the list view > Click on Save
      • Once a manually created list view has been saved, it displays in the Existing List View picklist 

Note: a manually created list view limited to Only I can see the list view is recommended for individual work and testing on the process.

 

Once you have defined general settings, source, and list view for the process, you can activate advanced options and/or activate audits on process runs : 

  •  

To activate Audits on process runs: 

  1. In the Audit Options window, click on the activation button

  1. In the Audit Name field, enter a free-form name 

  1. In the Audit Records, enter the number of lines on which to run the audit 

To activate Advanced Options:  

In Advanced Options window: 

 

  • Select the options you want to activate: 
    • Auto Merge Records (overwrites existing records with process results)
    • Salesforce Default Merge: check it to use the default Salesforce reparenting process, only works for SObject “Lead”, “Account”, “Contact” and “Case” objects.
        • /!\ the “Unmerge” will not be available.
    • Enable UnMerge Deduplication: allows to save simultaneously object on the parent record while merging; this procedure takes time and storage. If not activated you will not be able to undo the merge and the child’s non-unupdateable objects created on the parent record will be deleted.
    • Enable Reparenting Inactive Child : Allows to launch the reparenting on the Child records that contains ‘Inactive’ on the DQE_Status_c field. If not activated, all the Objects associated the Childs records containing ‘Inactive’ on the DQE_Status_c field, will be skipped.
    • Delete child records after merging: with this parameter, after processing a mergeing between the parents and child records, those records between the parents and child records, those records will be permanently deleted and not just flagged as “inactive”.
    • Authorized Empty Bulk Query : if checked, allow the process to run without returning “Failed” on the associated DQE Run if there is no data to be read.
  • Define the number of processed records per batch, in Maximum Record per Batchable, enter the amount of your choice (Records per batchable)
  • Limit the number of processed lines for test purposes, in Max Records Process (for each type), enter the amount of your choice (e.g. 10).
  • Launch an additional process when current process is run, in Trigger After Finish field, select the next process you want to launch.
  • Define the minimum number of percentage for an Auto Merge, move cursor to the chosen value in Minimum Threshold for Auto Merge .
  • Once your process has been set-up, select ‘Save’ or ‘Save and Run’ .
  • The newly created process displays in Processes Tab. 

 

 

Working on a Deduplication Process

 

A screenshot of a computer Description automatically generated with medium confidence

For a created process, available options and menus are: 

  • Run Process - launches or relaunches a process (uses your available credits) Run results are generated automatically. 
  • Stop Process -
  • Audit - generates the data report of the audit. 
  • Merge - merges and reparents your data using the mapping you chose when creating the process (see Creating a deduplication mapping with custom and/or standard fields). This reparents the objects you selected from the child objects to the new parent. 

If you did not select a mapping, there will be no reparenting.

  • Undo Merge: cancels your merge.

 

Creates Job from current setup > see Automating a Process 

Flush Results: deletes DQE results  

Filters: filters the DQE code results  

Open Dashboard: generates predefined dashboards which are included in DQE’s install package. 

Note: To create your own dashboards, use the standard reports and dashboards Salesforce procedures, and use the DQE results Data Model. 

(No action available) 

Refreshes data results 

Enables inline editing for unlocked fields. 

 

To view results and apply Salesforce filters, click on the List View name to open its dedicated screen > results display.

 

 

 

Editing an existing Deduplication Process

 

To modify the parameters of an existing deduplication process, you can:

  • Edit your process: the available options depend on the process you selected > see Creating a Deduplication Process
  • Delete your process.
  • Change Owner > select a user 

Note: editing or deleting a process does not delete related DQE’s results. For deleting results of an edited process, use the Flush Results icon . 

 

 

Editing a Deduplication Rule Set

You can edit existing deduplication rules from an existing Deduplication Process or from a new Deduplication Process. 

  • From an existing Deduplication Process: 
    • Flush Results you can delete the results you get from DQE.
    • Run Process to relaunch the deduplication process.  
  • From a new Deduplication Process > see Creating a Deduplication Process .

Related to

Related articles