Debugging hung runs and connection timeouts
A Data Flow run that connects over a Secure Shell (SSH) tunnel to a database such as PostgreSQL can get stuck with these symptoms:
- The run stays in Running indefinitely, with 0 bytes transferred and no output in the logs.
- Canceling and manually rerunning the same Data Flow succeeds within minutes.
- The issue typically affects only a small fraction of runs, clustered within a short window, such as your account's daily schedule tick.
These runs are commonly called zombie runs. In most cases, the root cause is on your network path, not in Data Integration. The checks in this guide let your infrastructure or database administration team confirm the cause and apply a fix without opening a support case.
Which layer is failing
A database connection that uses an SSH tunnel has two distinct points of failure. Understanding which one applies to your case determines which fix to apply.
SSH tunnel layer (bastion)
Either the bastion never establishes the SSH tunnel, or the connection between the bastion and the database silently drops. The ssh_tunnel_timeout_sec and ssh_tunnel_keepalive_sec connection settings cover this layer.
Database session layer
The tunnel is healthy, but the database accepts the socket and never responds. For PostgreSQL sources, the session-level connect_timeout setting and Transmission Control Protocol (TCP) keepalives on the extraction path cover this layer.
If your bastion is refusing or throttling new SSH connections, neither of the two protections above triggers. The connection never gets far enough to be instrumented. The checks in the next section are for this specific case.
Check for SSH connection throttling on the bastion
Run the following read-only commands on the bastion, in order, to determine whether it is throttling new SSH connections during your scheduled run window.
-
Check the configured limits:
sudo sshd -T | grep -iE 'maxstartups|maxsessions'
grep -iE '^(MaxStartups|MaxSessions|LoginGraceTime)' /etc/ssh/sshd_config -
Check the number of live concurrent SSH connections:
ss -tn state established '( sport = :22 )' | wc -l -
Watch the connection count during the scheduled burst window:
watch -n 2 "ss -tn state established '( sport = :22 )' | wc -l" -
Check the SSH logs for evidence of past throttling, using the command for your operating system:
-
Debian or Ubuntu:
sudo grep -i 'maxstartups' /var/log/auth.log* -
RHEL, Amazon Linux, or CentOS:
sudo grep -i 'maxstartups' /var/log/secure* -
Journald-based systems, scoped to the incident window:
sudo journalctl -u sshd --since "<start>" --until "<end>" | grep -iE 'maxstartups|refused|too many'
-
A line such as sshd[nnnn]: error: beginning MaxStartups throttling confirms that the bastion randomly refused new connections at that time.
Remediation: raise the bastion's MaxStartups limit
OpenSSH's default MaxStartups value is 10:30:60. This means the bastion accepts the first 10 unauthenticated connections freely, then randomly refuses a rising percentage of new connections, up to a hard ceiling of 60 simultaneous unauthenticated connections. An account that opens several hundred tunnels within a few-minute scheduled window can trip this limit.
-
Validate the current configuration before making changes:
sudo sshd -t -
Open
/etc/ssh/sshd_configin a text editor and update the following values:MaxStartups 30:50:200 # OpenSSH default is 10:30:60
MaxSessions 20 -
Reload the SSH daemon to apply the change:
sudo systemctl reload sshdIf
systemctlis not available, usesudo service sshd reloadinstead.
Other causes to rule out
If the bastion checks come back clean, also check for:
- PostgreSQL connection drops, restarts, failovers, or idle and statement timeouts around the time of the hang.
- Network or firewall idle timeouts between the bastion and the database.
- Bastion load, configuration, or network changes around the time of the hang.
What to send Support if the checks come back clean
If none of the checks above explain the hang, open a support case and include:
- The Data Flow name, environment, run ID, and hang timestamp (UTC).
- The output of the
sshd -Tand log-grep commands from this guide for the incident window. - Whether a manual rerun of the same Data Flow succeeded.