z39_gate processes won't start

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Article Type: General
Product: Aleph
Product Version: 20

Description:
On Feb. 10th, we installed the latest ver. 20 service pack (3389) on our pre-production server. On Feb. 18th and again on Mar. 12th, we had several z39_gate processes that would not start on our pre-production server. We had to restart all of Aleph both times to fix the problem. Below is the error message in the log files:

07:34:27-12/03 /exlibris/aleph/a20_1/aleph/exe/z39_gate [fatal] bind [Address already in use]

Have there been any problems like this reported from the service pack?

from netstat -anp:

tcp 0 0 192.168.6.136:9617 192.168.6.138:1521 ESTABLISHED -
tcp 0 0 192.168.6.136:9616 192.168.6.138:1521 ESTABLISHED -
tcp 0 0 192.168.6.136:9627 192.168.6.138:1521 ESTABLISHED -
tcp 0 0 192.168.6.136:9625 192.168.6.138:1521 ESTABLISHED -
tcp 0 0 192.168.6.136:9624

--------------------------------------------------------------------------------

They are actually cobol processes that are using those source ports – They are connecting to Oracle:

(snippet from netstat)
tcp 0 0 192.168.6.136:9618 192.168.6.138:1521 ESTABLISHED 21655/rts32
tcp 0 0 192.168.6.136:9617 192.168.6.138:1521 ESTABLISHED 21655/rts32
tcp 0 0 192.168.6.136:9616 192.168.6.138:1521 ESTABLISHED 21656/rts32
tcp 0 0 192.168.6.136:9627 192.168.6.138:1521 ESTABLISHED 22054/rts32
tcp 0 0 192.168.6.136:9625 192.168.6.138:1521 ESTABLISHED 22054/rts32
tcp 0 0 192.168.6.136:9624 192.168.6.138:1521 ESTABLISHED 22055/rts32

Looks like the ABC UE 01 job:

# ps -ef | grep -i 21655
aleph 21655 1 0 05:01 ? 00:00:00 /exlibris/aleph/a20_1/aleph/exe/rts32 ue_01_a BEC30.a20_1

# ps -ef | grep -i 21656
aleph 21656 1 0 05:01 ? 00:00:00 /exlibris/aleph/a20_1/aleph/exe/rts32 ue_01_word_parallel BEC30.a20_1

# ps -ef | grep -i 21654
root 16230 15907 0 09:42 pts/2 00:00:00 grep -i 21654

Resolution:
[From site:]

We believe we have found the cause.

This file: /etc/sysctl.conf contains parameters at the end of the file:
# Exlibris Parameters
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
# End of Exlibris Parameters

These parameters were added by the script 'updateRHSysParams.bash'; this script was called by the AIK and the action is seen in the aik log:
[root@tulip log]# pwd
/exlibris/ftp_from_exlibris/aik/log

[root@tulip log]# cat aleph_installation_kit.log | grep local_port
Updating parameter net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.ip_local_port_range = 9000 65500

The default Red Hat range is 32678 65500; we are not clear why the range needs to be changed from the default, but the setting does conflict with our use of the 9600-9629 range of ports for the z39_gates.

We will comment out this parameter on all the servers and implement the change interactively on the servers.

[Later: No further problems after implementing this change.]

Article last edited: 10/8/2013