SFX to Alma AutoExtract Migration

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

The objectives of the SFX to Alma migration process are to migrate SFX data to the Alma environment quickly and efficiently, eliminate any coordination and scheduling issues with Ex Libris, and enable the independent extraction of data.

The functioning of SFX is not affected by this process running in the background, and your normal work in SFX can proceed in parallel while it is running. However, we recommend running it during non-peak hours for best performance.

This document explains how to download and install the SFX to Alma Migration Package, perform the extraction SFX data, and transfer it securely to Ex Libris’ SFTP server.

The migration package kit supports Both CDI and non-CDI customers. CDI customers must install SFX version 4.10.15 or above before moving to Alma.

If you have any questions, contact Ex Libris.

Requirements

Standard migration requirements documented in Getting Ready for Alma and Discovery Implementation.
SFX needs to be on v4.1.2 (Nov 2012) or above to migrate to Alma, plus the most recent KB version.
Make sure that you have enough CPU capacity and that the RAM is not using swap memory. If the server is overloaded, scale it up before attempting to run the extract.
Open an access in your firewall to the Ex Libris SFTP server (MFT) in order to transfer the extracted data:
- APAC customers: migration-dc05.hosted.exlibrisgroup.com (APAC-DC05) or migration-dc07.hosted.exlibrisgroup.com (APAC-DC07). The relevant SFTP should be supplied by your Professional Services.
- EU customers: migration-dc03.hosted.exlibrisgroup.com (EU-DC03) or migration-dc06.hosted.exlibrisgroup.com (EU-DC03) . The relevant SFTP should be supplied by Professional Services.
- NA customers: migration-dc01.hosted.exlibrisgroup.com (NA-DC01) or migration-dc04.hosted.exlibrisgroup.com (NA-DC04) . The relevant SFTP should be supplied by Professional Services.
- CANADA customers: migration-dc82.hosted.exlibrisgroup.com
- CHINA customers: migration-dc81.hosted.exlibrisgroup.com.cn
All servers listed above should use SFTP port 10022.

Verify that you have at least 2 GB of free disk space.

Overview of the AutoExtract Process

The following steps describe the SFX AutoExtract process:

An email with the following information is sent to you by Ex Libris:
- A link to the AutoExtract package path on FTP server
- Credentials for FTP download
Download the AutoExtract package from the FTP server.
Install the AutoExtract package on your SFX server.
Run the extraction of your SFX data using the AutoExtract package.
Transfer the extracted data to your regional Ex Libris SFTP server.

The server used to retrieve the SFX2Alma package is an FTP server; the server where you send your extracted data is a different server, which uses SFTP.

These steps are described in detail below.

Downloading and Installing the AutoExtract Package

The following section describes how to download and install the AutoExtract package.

To download and install the AutoExtract package:

Log on to your local SFX instance on your SFX server (for example, sfxlcl41, or other local instance).
Enter the following command to create a directory in which to store the zip file:
>> mkdir /exlibris/ftp_from_exlibris/sfx2alma_migration

You may need to create the ftp_from_exlibris subdirectory first, i.e.:

>> mkdir /exlibris/ftp_from_exlibris
Download the package from Ex Libris MFT server as described in the email sent from Ex Libris.
Enter the following command to change the directory as the local user to the sx directory:
>> sx

(Or: >> cd $SFX_HOME/$SFX_INST/)
Move the package SFX2Alma.zip from /exlibris/ftp_from_exlibris/sfx2alma_migration to the sx directory of the local SFX user.
>> cp /exlibris/ftp_from_exlibris/sfx2alma_migration/SFX2Alma.zip .

Note: sometimes this triggers a question: cp: overwrite ‘./SFX2Alma.zip’? to which you should answer Y
Enter the following command to unpack the zip file:
>> unzip SFX2Alma.zip
Run the following command:
csh -f unzip_package.sh

This unpacks the AutoExtract package and puts its constituents into their designated places in the SFX installation.

If you want to run the auto extract process several times (for test and for productive load for example, check if there is a new version of the package available. If so, proceed with the new package as described above.
If you get the message when running the unzip command, answer with A to replace the files from the old package with the ones from the new package.

replace admin/kbtools/sfx2alma_migration.pl? [y]es, [n]o, [A]ll, [N]one, [r]ename:

Extract using the Menu

The steps of the AutoExtract package are driven by a menu.

To run the menu:

Log on to your local SFX instance as 'local SFX user' on your SFX server.
Enter the following command to move to the directory with the migration menu:
>> da

(or:>> cd $SFX_HOME/$SFX_INST/admin/)

>> cd kbtools
Enter the following command to start the menu:
>> ./SFX_MIGRATION_MENU

Verify your institution code and institution ID.

If you are migrating multiple SFX instances into a single Alma institution, you should have the same institution code and institution ID for all SFX exports.

The menu is displayed:

SFX Migration Menu
    SFX base directory : /exlibris/sfx_ver/sfx4_1
    SFX Instance code : sfxlcl41
1. Extract SFX data
2. Transfer extracted data
0. Exit
Please select [0]:

Extracting and Converting the Data

To extract your SFX data and convert it into an Alma supported format:

Select 1. Extract SFX data to retrieve the data from the SFX database.
The following appears:

Did you consult with an Ex Libris representative concerning loading SFX data with Available For groups to Alma? (Y/N) [N]
If you have a multi-institute SFX structure (see Multiple institutes) to be extracted enter Y.
If you ran the extraction before and data is still in the export directory, the following appears:

There appears to be data from a previous extraction in the directory.

If you continue, this old data will be deleted.

Do you want to continue? (y/n): [n]
Enter y to delete the data and continue with the extraction process. Enter n, to stop the extraction process and move the old data to a different location, if you do not want to delete it.
Select Default institute unless you are a multi-institute site.

Multiple Institutes

When running the above option within a multi-organizational (institutes/campuses) environment, a menu similar to the one below is displayed that allows you to specify the institution or group for which the migration should be carried out.

Institutes and/or groups have been defined in the instance.
Choose institute or group for which the migration file should be created:
Multiple selections can be done with a comma separator.
1 All institutes defined in the instance
2 All institutes and groups defined in the instance
3 Default institute
4 walnut
5 peanut
6 almond
Please select [1-6,q]:

After choosing the relevant numbered option/s, the process begins.

Extract using the Command Line

Customers with very large institutes may choose to extract using the command line.

Extract the Default Institute

cd kbtools

nohup perl sfx2alma_migration.pl --action=both --institute=Default > & ../../logs/sfx2alma_migration.pl-nohup-<datetime>.log &

You can name the log file anything you like, but we recommend to use a date/time stamp in case of multiple attempts.

Extract Multiple SFX Institutes at the same time

Run this extract after you extract the default institute.

First, find the last default collection update file

cd dbs/scratch/e_collection_update/

ls -lrt free-e-collection-default.* |tail -1

You will see a filename like this

-rw-rw---- 1 sfxlcl4 sfx 370263138 Jun 28 18:48 e-collection-default.20210626154507

Then, extract from SFX using that file name that you saw in the previous step. In addition, specify the institutes you wish to export.

cd kbtools

nohup perl sfx2alma_migration.pl --default_institute_filename=e-collection-default.20210626154507 --action=both --institute=YYY --institute=ZZZ > & ../../logs/sfx2alma_migration.pl-nohup-Institute-<datetime>.log &

where YYY and ZZZ are institute codes on your SFX server. You can name the log file anything you like, but we recommend to use a date/time stamp in case of multiple attempts.

Monitoring the Extraction Process

The log is displayed on the screen and is saved in the following location:

$SFX_HOME/$SFX_INST/logs/

sfx2alma_migration.pl-<yyyy>-<mm>-T<hh>:<mm>:<ss>.log</ss></mm></hh></mm></yyyy>

The process displays the following message:

Start exporting active object portfolios in MARC XML format

f you get the following message, the extraction process has completed successfully.

Started packaging files
  Packaging files to: sfx2alma_migration_r.49EXL_sfxlcl41_20151100_2015-03-20_10-50-19.zip
  Packaging files to: sfx2alma_migration_r.49EXL_sfxlcl41_20151100_2015-03-20_10-50-19.tar
  Removing files
Finished packaging files
SFX data extraction finished
Press any key to continue

If you encounter any problems, contact your Ex Libris project manager.

Monitoring from the command line

In addition to monitoring the log as described above, you can check if the extract process is still running by entering this at the command line:

ps -ef | grep sfx2alma

If the process is still running, you will get a response similar to this:

sfxtul 1218 32362 0 22:36 pts/0 00:00:02 perl sfx2alma_migration.pl --action=both --institute=Default

You can also monitor the log file(s).

ls -1rt sfx2alma*

The log files are as follows

sfx2alma_migration.pl-nohup-Default-20210626.log - specified output from the command line in Default extract

sfx2alma_migration.pl-2021-06-26T15:45:03.log - output from the SFX AutoExtract program in Default extract

sfx2alma_migration.pl-nohup-LBL-20210627.log - specified/output from the command line in institute LBL extract

sfx2alma_migration.pl-2021-06-27T04:03:01.log - output from the SFX AutoExtract program in institute LBL extract

Transferring the Extracted Data to Ex Libris

At the end of the extraction, the processed data can be found in the directory on the SFX server:

$SFX_HOME/$SFX_INST/export/

At this point, the extracted data is ready for uploading to the Ex Libris SFTP server. Verify that a connection is open to your region’s SFTP server before uploading.

To upload the extracted data to the Ex Libris SFTP:

Select 2. Transfer extracted data from the SFX Migration Menu.
Select the SFTP server to which the data will be transferred, according to your region:

Select the ftp server by the optional regions:

1. APAC

2. EUR

3. USA

4. CANADA

5. CHINA

Please select your region:

If your region is APAC/EU/NA, you are asked to select one of the data centers in your region, according to Professional Services.
You are asked to enter your sftp user name. Copy and paste the key from the link sent to you by Ex Libris. Below is an example of the key you are asked to insert.

Enter your password:

-----BEGIN RSA PRIVATE KEY-----

MIIEoAIBAAKCAQEAz+h+RByvWo5aLNxpXS0hNh2DKlipYA81Ur+pVzoILXiA3EWW

h6zfCOxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxbP0UPtbKojfhIgCbAJrtwa7jszxu

XW9vn95ulDUNNuoyKGyyVeve7InsFx0NTzlLaJeJq3vfPCVXvO8InRY7giX/zCE0

yxsOal+Wsf5vn+9L8XGEdcaG37vPWGEBfaNYywIBIwKCAQBNORj0ujKAuIfkxuVO

fnoMySIXCv0VDPaMcxMDJC7sUVRvEoheR4YKnIVxq+edr12hLcdp7g5VjihxmxVP

-----END RSA PRIVATE KEY-----

The following message is displayed:

Verifying the connection to the sftp server requested can be established. Please wait.

If the connection cannot be established, the following message is displayed:

Cannot establish connection to the sftp server requested. Do you want to try again resetting your password? (Y/N) [N]

Inform Ex Libris when the data transfer process is complete.

The data is transferred securely using SFTP.
The data transfer process can take up to several hours to complete, depending on the size of your data and your Internet connection speed. The functioning of SFX is not affected by this process running in the background, and your normal work in SFX can proceed in parallel. If the process does not complete within 24 hours, contact Ex Libris.

Troubleshooting

Copying the MFT SSH_KEY

In the extract process, you are asked to copy the ssh_key given to you by your Ex Libris representative. If you do not copy the entire key, the data is not packaged and transferred. If this situation occurs, do the following:

Close the extract menu.
Copy the entire content of the ssh_key (including the first and last "BEGIN"/"END" lines) and paste it into a text file in Windows.
Save the file with the following name: your generated sftp UserName + _urm_sftp_ssh_key (without any suffix). For example, if your sftp user name = MyGenUsrName, the ssh_key file is named: MyGenUsrName_urm_sftp_ssh_key.
Copy the ssh_key file to the SFX server in the location where the SFX2Alma kit was placed in the ../admin/kbtools directory.
Open a putty session to your SFX server, and go to the ../admin/kbtools directory. Set permissions to your ssh_key file: chmod 600 MyGenUsrName_urm_sftp_ssh_key.
Rerun the extract menu. This time you are not asked to insert the ssh_key.