GCP VM — Backend server installation

1. Architecture

The DQE One Server and all its dependencies are packaged as Docker images and run with Docker Compose on a Google Compute Engine VM. Once deployed, the server pairs with a Salesforce org in which the DQE One package is installed (available on the AppExchange).

This document describes a possible configuration, either with the VM directly exposed over HTTPS, or behind a GCP load balancer. It does not cover the security layer of your own environment.

Flow Diagram

The diagram below describes all the inbound/outbound flows as well as the IPs and ports requested by the application.

Security Measures

We cannot provide an example of how to implement the security layer that would fit perfectly into your architecture. However, we can provide some recommendations that are relevant to the use of the app itself.

• Protocols and ports: All outgoing/incoming flows from the application go through the HTTPS protocol (port 443). There is no need to open another port to the internet. The DQE IPs that must be accessible are described in the preceding diagram.
• IP filtering: The DQE One Server only needs to be accessible from Salesforce servers. It is therefore recommended to filter incoming IPs, with a VPC firewall rule or a Cloud Armor policy. The exception is the CustomUI interface, which must be accessible from your users' web browsers: allow your company's public IPs or VPN as well. See the list of IPs published by Salesforce (don't forget the Hyperforce IPs).
• Authentication protocol: Salesforce natively offers several protocols to secure requests between its servers and the DQE One Server. When configuring the "DQE One Customer" Connected App in your org, you can select the "Basic Auth" authentication method and configure your application firewall accordingly.

If the VM is directly exposed to the internet, it must have a DNS entry and an associated SSL certificate that point to it.

If the VM is behind a GCP load balancer, the DNS entry and the SSL certificate are installed on the load balancer, which forwards the traffic to the VM inside your VPC.

Recommendation

This section lists the components required to install the DQE One Server instance on GCP.

Note: Estimates are based on 1 million records. Depending on the volume of the databases processed, it may be necessary to increase the memory to optimize processing times.

• Compute Engine VM: Ubuntu 22.04 or 24.04 LTS recommended, for example machine type e2-standard-4 (4 vCPU / 16 GB).
• Load balancer: not required, but can be used if already available.

Note: the queue worker alone can use up to 5 GB of memory during processing. For more information on VM instances, see the Google Cloud documentation.

Composition and services

The DQE One Server exposes the services and APIs called by Salesforce through the DQE One app. It is made of the following services, all defined in the docker-compose.yml file described in 2.4.

Web application (web)

A backend container that contains a microservices application. This application exposes all the APIs called to launch processes, manage processing queues and instantiate processing workers. It depends on Redis and RabbitMQ. It is exposed on the web through Nginx only, under the /unify/ path.

• image: provided by DQE Software and hosted on the DQE Azure Container Registry:

  <Name of the registry container>.azurecr.io/<name of the image>

• environment: SFAPIVERSION, PORT, REDIS_URL, CLOUDAMQP_URL, CUSTOMUI, UNIFYSERVERURL
• command: bash ./entrypoint.sh

Worker (worker)

A backend container that processes queued jobs asynchronously. It consumes tasks from the RabbitMQ queue and executes them. It uses the same image as the web application and is not exposed on the web.

• image: same image as the web application
• environment: SFAPIVERSION, WORKDIRPATH, REDIS_URL, CLOUDAMQP_URL
• command: python -u ./unify/queue_worker.pyc

CustomUI (customui)

A web application exposing the interface used to configure deduplication rules. It must be accessible from your users' web browsers, through Nginx at the root path /. It also calls some backend APIs, for example to retrieve metadata from the CRM.

• image: provided by DQE Software
• environment: PORT, SESSION_SECRET

Redis (redis)

A key/value database used to store internal operating keys such as Salesforce org configurations, sessions, etc.

You can use the official redis image. As a security measure, some organizations block public images and only allow images from private registries: in this case, use the compatible Redis image published by DQE on its private registry.

• image: redis:7.4-alpine or the DQE image
• volumes: redis_data:/data (Docker named volume)

RabbitMQ (rabbitmq)

A queue manager that receives and schedules processing requests. Until a process has been assigned to a worker and completed, it is parked in the queue. This also provides resiliency: if the server is restarted, processing resumes where it left off.

As with Redis, you can use the official image or the one on the DQE private registry.

• image: rabbitmq:3.13-management-alpine or the DQE image
• volumes: rabbitmq_data:/var/lib/rabbitmq (Docker named volume)
• environment: RABBITMQ_DEFAULT_USER, RABBITMQ_DEFAULT_PASS, RABBITMQ_DEFAULT_VHOST

2. Installation

Two scenarios are possible:

• Scenario 1 — VM directly exposed: the VM has an external IP, and Nginx on the VM handles HTTPS with your SSL certificate.
• Scenario 2 — VM behind a GCP load balancer: an external Application Load Balancer handles HTTPS and forwards HTTP traffic to the VM inside your VPC. The VM does not need an external IP.

The steps are the same for both scenarios, except the Nginx configuration and the firewall rules.

2.1. Prerequisites

Launch the VM and connect to it, for example with SSH from the Compute Engine console or with gcloud compute ssh. For more details, see the Google Cloud documentation.

Docker installation

Install Docker Engine and the Docker Compose plugin from the official Docker repository:

sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

Log out and log in again so that your user can run Docker without sudo, then check the installation:

docker --version
docker compose version

Docker Compose

Docker Compose is installed as a plugin in the previous step: use docker compose (with a space). The standalone docker-compose binary is no longer needed.

NGINX installation

sudo apt-get update
sudo apt-get install -y nginx
sudo mkdir -p /etc/nginx/ssl

Create the file /etc/nginx/sites-available/dqe-one with the content matching your scenario.

Scenario 1 — VM directly exposed over HTTPS. Copy your SSL certificate and its private key to /etc/nginx/ssl/ first:

server {
    listen 80;
    server_name <your-domain.com>;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name <your-domain.com>;

    ssl_certificate     /etc/nginx/ssl/<certificate>.pem;
    ssl_certificate_key /etc/nginx/ssl/<private-key>.key;
    ssl_protocols       TLSv1.2 TLSv1.3;

    access_log /var/log/nginx/dqeone.log;
    error_log  /var/log/nginx/dqeone-error.log error;

    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 300s;

    # DQE One UI (CustomUI)
    location / {
        proxy_http_version 1.1;
        proxy_set_header   Upgrade    $http_upgrade;
        proxy_set_header   Connection "upgrade";
        proxy_pass         http://127.0.0.1:8001/;
    }

    # DQE One Server API — the trailing "/" removes the /unify/ prefix
    location /unify/ {
        proxy_pass http://127.0.0.1:8000/;
    }

    location /lookup {
        proxy_pass http://127.0.0.1:8000;
    }

    location /static {
        proxy_pass http://127.0.0.1:8000;
    }

    location /API {
        proxy_pass http://127.0.0.1:8000;
    }
}

Scenario 2 — VM behind a GCP load balancer. HTTPS is handled by the load balancer:

server {
    listen 80;

    access_log /var/log/nginx/dqeone.log;
    error_log  /var/log/nginx/dqeone-error.log error;

    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_read_timeout 300s;

    # DQE One UI (CustomUI)
    location / {
        proxy_http_version 1.1;
        proxy_set_header   Upgrade    $http_upgrade;
        proxy_set_header   Connection "upgrade";
        proxy_pass         http://127.0.0.1:8001/;
    }

    # DQE One Server API — the trailing "/" removes the /unify/ prefix
    location /unify/ {
        proxy_pass http://127.0.0.1:8000/;
    }

    location /lookup {
        proxy_pass http://127.0.0.1:8000;
    }

    location /static {
        proxy_pass http://127.0.0.1:8000;
    }

    location /API {
        proxy_pass http://127.0.0.1:8000;
    }
}

Enable the configuration:

sudo ln -s /etc/nginx/sites-available/dqe-one /etc/nginx/sites-enabled/dqe-one
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t
sudo systemctl reload nginx

2.2. VPC firewall rules

Run these commands from Cloud Shell or any machine with the gcloud CLI.

Scenario 1: allow HTTP/HTTPS from Salesforce IP ranges and your company's IPs only:

gcloud compute instances add-tags <vm-name> --zone <zone> --tags dqe-one

gcloud compute firewall-rules create dqe-one-https \
  --network <vpc-name> \
  --direction INGRESS \
  --allow tcp:80,tcp:443 \
  --source-ranges <Salesforce IP ranges>,<admin VPN IP range> \
  --target-tags dqe-one

Scenario 2: allow HTTP from the Google load balancer and health check ranges only. Filter Salesforce and company IPs with a Cloud Armor policy on the load balancer.

gcloud compute instances add-tags <vm-name> --zone <zone> --tags dqe-one

gcloud compute firewall-rules create dqe-one-lb \
  --network <vpc-name> \
  --direction INGRESS \
  --allow tcp:80 \
  --source-ranges 130.211.0.0/22,35.191.0.0/16 \
  --target-tags dqe-one

For scenario 2, create an external Application Load Balancer with an HTTPS frontend (your SSL certificate), a backend service pointing to an instance group that contains the VM on port 80, and an HTTP health check on path /. For more details, see the Google Cloud documentation.

Note: ports 8000 and 8001 are bound to 127.0.0.1 in the docker-compose.yml below: they are only reachable through Nginx and must not be opened in the firewall.

2.3. Configuration files

Create the application folder and the .env file with the values provided by DQE Software:

sudo mkdir -p /opt/dqe-one
sudo chown $USER /opt/dqe-one
cd /opt/dqe-one
nano .env
nano docker-compose.yml

Content of the .env file. Use strong passwords, without the characters @ : / ? # (the RabbitMQ password is inserted into a connection URL):

# Values provided by DQE Software
SERVER_IMAGE=<registry-url>/<dqe-one-server-image>:<tag>
CUSTOMUI_IMAGE=<registry-url>/<dqe-one-ui-image>:<tag>
REDIS_IMAGE=redis:7.4-alpine
RABBITMQ_IMAGE=rabbitmq:3.13-management-alpine

SFAPIVERSION=v65.0

# Strong passwords, without @ : / ? # (inserted into the AMQP URL)
RABBITMQ_USER=dqeone
RABBITMQ_PASSWORD=ChangeMe_Str0ngPassword
SESSION_SECRET=ChangeMe_LongRandomString

2.4. DQE One Server — docker-compose.yml

Content of the docker-compose.yml file:

name: dqe-one

x-logging: &default-logging
  driver: json-file
  options:
    max-size: "50m"
    max-file: "5"

services:
  customui:
    image: ${CUSTOMUI_IMAGE}
    container_name: custom-ui
    restart: unless-stopped
    logging: *default-logging
    environment:
      - PORT=8001
      - SESSION_SECRET=${SESSION_SECRET}
    ports:
      - "127.0.0.1:8001:8001"     # reachable only through Nginx

  web:
    image: ${SERVER_IMAGE}
    container_name: one-server
    hostname: oneserver
    restart: unless-stopped
    logging: *default-logging
    depends_on:
      redis:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy
    environment:
      - SFAPIVERSION=${SFAPIVERSION}
      - PORT=8000
      - REDIS_URL=redis://redis:6379
      - CLOUDAMQP_URL=amqp://${RABBITMQ_USER}:${RABBITMQ_PASSWORD}@rabbitmq:5672/admin
      - CUSTOMUI=http://customui:8001
      - UNIFYSERVERURL=http://web:8000
    ports:
      - "127.0.0.1:8000:8000"     # reachable only through Nginx
    command: bash ./entrypoint.sh

  worker:
    image: ${SERVER_IMAGE}
    container_name: queue-worker
    restart: unless-stopped
    logging: *default-logging
    depends_on:
      web:
        condition: service_started
      rabbitmq:
        condition: service_healthy
    environment:
      - SFAPIVERSION=${SFAPIVERSION}
      - WORKDIRPATH=/app/unify
      - REDIS_URL=redis://redis:6379
      - CLOUDAMQP_URL=amqp://${RABBITMQ_USER}:${RABBITMQ_PASSWORD}@rabbitmq:5672/admin
    command: python -u ./unify/queue_worker.pyc

  redis:
    image: ${REDIS_IMAGE}
    container_name: redis
    hostname: redis
    restart: unless-stopped
    logging: *default-logging
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 10

  rabbitmq:
    image: ${RABBITMQ_IMAGE}
    container_name: rabbitmq
    hostname: rabbitmq
    restart: unless-stopped
    logging: *default-logging
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASSWORD}
      - RABBITMQ_DEFAULT_VHOST=admin
    volumes:
      - rabbitmq_data:/var/lib/rabbitmq
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
      interval: 10s
      timeout: 10s
      retries: 12
      start_period: 30s

volumes:
  rabbitmq_data:
  redis_data:

Key points:

• Passwords and image names are read from the .env file.
• Redis and RabbitMQ must be healthy before the web application starts, and all containers restart automatically after a VM reboot.
• Logs are rotated (5 × 50 MB per container) so that they cannot fill the disk.

3. Launcher

Although the server is deployed on GCP, the DQE Docker images are provided through an Azure Container Registry managed by DQE Software. No Azure tool is needed: Docker logs in directly.

3.1. Connecting to the DQE Azure Container Registry

docker login <registry-url> --username <Login provided by DQE>

Enter the password provided by DQE when prompted. A successful login displays Login Succeeded.

3.2. Launch of docker-compose.yml

cd /opt/dqe-one
docker compose pull
docker compose up -d
docker compose ps

Expected output — all services Up, Redis and RabbitMQ healthy:

NAME           SERVICE    STATUS
custom-ui      customui   Up
one-server     web        Up
queue-worker   worker     Up
rabbitmq       rabbitmq   Up (healthy)
redis          redis      Up (healthy)

If a container is restarting, check its logs (replace web with the service name):

docker compose logs -f web

Finally, open https://<your-domain.com> from a browser to check that the DQE One interface is reachable. In scenario 2, also check that the backend is Healthy in the load balancer.

3.3. Update

When DQE Software provides a new version, update the image tags in the .env file, then run:

cd /opt/dqe-one
docker compose pull
docker compose up -d