Aleph to Alma AutoExtract Migration

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

The objectives of the Aleph to Alma migration process are to migrate Aleph data to the Alma environment quickly and efficiently, eliminate any coordination and scheduling issues with Ex Libris, and enable the independent extract of data.

This document explains how to download and install the Aleph to Alma migration package, perform the extract and transformation of Aleph data, and transfer it to Ex Libris’ secure file server (MFT).

This document discusses migrating from Aleph 18.01 and later.

Requirements

In addition to the standard migration requirements documented in Getting Ready for Alma and Discovery Implementation, there are additional requirements for the AutoExtract process.

The migration process distinguishes between two migration events -- Testload and Cutover.

Testload is done a few months before the actual migration, so that the customer can become familiar with Alma and be able to prepare the library for migration. Cutover is the actual migration.

There are some differences in the procedures and processes between Testload and Cutover. They are specified throughout the document. Three weeks before Testload and two weeks before Cutover you are sent the following:

Migration Tool path on MFT (secure file) server
Credentials for MFT access for file download
Links to the following documents:
- Aleph to Alma Migration Guide
- Getting Ready for Alma and Discovery Implementation
- This document

Provide your filled-in migration form to your Ex Libris project manager for approval. See the following sections and the Aleph to Alma Migration Guide describe for how to generate the migration form and fill it in. Make sure the e_resource file for P2E is prepared as documented in Getting Ready for Alma and Discovery Implementation.

Open an access in your firewall to the Ex Libris Secure File server (MFT) in order to transfer the extracted data:

APAC customers: migration-dc05.hosted.exlibrisgroup.com (APAC-DC05) or migration-dc07.hosted.exlibrisgroup.com (APAC-DC07). The relevant connection info should be supplied by your Professional Services.
EU customers: migration-dc03.hosted.exlibrisgroup.com (EU-DC03) or migration-dc06.hosted.exlibrisgroup.com (EU-DC03) . The relevant connection info should be supplied by Professional Services.
NA customers: migration-dc01.hosted.exlibrisgroup.com (NA-DC01) or migration-dc04.hosted.exlibrisgroup.com (NA-DC04) . The relevant connection should be supplied by Professional Services.
CANADA customers: migration-dc82.hosted.exlibrisgroup.com
CHINA customers: migration-dc81.hosted.exlibrisgroup.com.cn

All servers listed above should use port 10022.

Verify that you have enough space.

Tablespace - Our recommendation for TS1 is 2 GB per million bibliographic records.
Disk space - Each million bibliographic records expected to be extracted require 10 GB of disk space. If you contracted for ADAM migration (digital objects), add to this calculation the ADAM current diskspace * 2, for example:
- Expected number of bibliographic records = 4 Million
- ADAM objects DiskSpace = 3GB
- Total required diskspace = (4*10) + (3*2) = 46GB

Downloading and Installing the AutoExtract Tool

Note that it is mandatory to download and install the AutoExtract kit from scratch twice during the migration period. the first time before the test load and the second time before the cutover. This is to ensure that cases reported after the test load are handled in the AutoExtract kit.

To download and install the AutoExtract package:

Log in as the aleph user.
Create the following directory for storing the zip file you download (if re-installing the tool is needed, first clean the path):
>> mkdir /exlibris/zip_from_exlibris/aleph2alma_migration
Download the tool from the MFT server, as described in the email sent from Ex Libris. Unzip the package
>> unzip Aleph2Alma.zip
Move to the desired ADM library, for example:
>> dlib xxx50
Run the utility
>> aleph2alma_menu

The main menu is displayed:

Aleph2Alma Migration Tool - Main Menu
=====================================
A. Install the Aleph2Alma Migration Tool
B. Generate the initial Aleph Migration Form
C. Validate the P2E file
D. Extract the Aleph Data
E. Check Status
F. Package the Extracted Data
G. Transfer the Packaged Data
H. Extract Historical Loans
Select A Install the Aleph2Alma Migration Tool.
Confirm the installation execution.
Start installation? (Y/N) [N]: Y
Confirm the ADM.
You are connected to the Adm xxx50. Is this the correct library? (Y/N) Y

If this is not the ADM library that you want to extract, answer No, exit the menu, and use the dlib command to move to the ADM library that you want.

After confirming the ADM library, the setup begins:

Installing Aleph2Alma migration tool. This may take several minutes. Please wait.

Do not exit while the setup is running or you will have to re-install the tool. If you encountered a problem in the installation process, contact Ex Libris.

Generate and Validate the Initial Aleph Migration Form

To generate an initial Aleph Migration form:

Select option B in the main menu. The following is displayed:
Generate and validate the initial Aleph Migration Form
======================================================
1. Create the Aleph Migration Form
2. Validate the Aleph Migration Form
3. Create a Dummy Migration Form
Please select an option [exit]:
Select option 1. (For information regarding option 3. Create a Dummy Migration Form, see Create a Test Migration Form and Run the Extract).
Confirm the library. The following message is displayed:
Generating the initial Aleph Migration Form. This may take several minutes. Please wait.
Still running ...

When generating the Aleph Migration Form, if the AlephMigrationForm.xlsm migration form file already exists in the alephe_tab directory, the extract kit generates the new migration form already filled in with the answers from the old form. This can be useful before cutover when generating the new migration form. The questions and answers can be generated from the testload form.

The generated form, AlephMigrationForm.xlsm, can be found in the active $alephe_tab directory.
Staff Librarians should fill in the downloaded migration form.
The first validation of the migration form is done after staff librarians fill in the migration form, using the migration form Validation Tool. For more information, see Validating the ILS Migration Form. The first validation step checks for consistency in question answering and making sure all mandatory questions are answered.
The second stage of migration form Validation is done on the Aleph server, before the extract by the system librarian. The second validation stage compares the responses in the form with the values in the Aleph database. To begin the second step of Aleph migration form validation, upload the migration form to your active $alephe_tab directory. Open the menu again.
(dlib xxx5x, aleph2alma_migration).
Select option B in the main menu and then option 2, in order to validate the migration form.
Confirm the library:
Validate the initial Aleph Migration Form
-----------------------------------------
You are connected to the Adm MAN50. Is this the correct library? (Y/N) Y
The Aleph Migration Form validation is running. Please wait...

If the form is filled in correctly the validation ends successfully. If there are errors, you are referred to a log describing the errors.

Validate the P2E File

To validate the P2E file:

Create the file according to the instructions in the Aleph to Alma Migration Guide.
Place the file under the active $alephe_tab directory and name it: e_resource_input.csv
Run the kit and Select option C from the main menu.
Correct any errors regarding the format or the electronic type in the file, for example:
- Format error of bibliographic key in line 1: 12345-ABC01
- Not allowed type in line 2: Xortfolio
- Multiple mapping for BCL01000513913 in line 5: [DB, Portfolio]

Setup and Data Extract

To perform the setup and data extract:

Place the approved migration form under the active $alephe_tab directory and name it:
AlephMigrationForm.xlsm.
Place the validated P2E file under the active $alephe_tab directory and name it:
e_resource_input.csv
Select option D in the Aleph to Alma utility menu The following menu is displayed:
Please select the data required:
1. All areas
2. Fulfillment only (Patron/Request/Loan)
Please select an option [exit]:
Select the data required. If the status of the migration is not fulfillment cutover, select option 1.
If you select option 2, the following is displayed:
Only Patron/Request/Loan will be extracted. Correct? (Y/N)
Enter Y or N.
If you answer Y, only the circulation area (patrons/loan/requests) is extracted.

If you answer N or press Enter, you return to the main menu.
Confirm running the extract.
Start the extract? (Y/N) [N]: Y
Confirm the following:
- ADM library
  You are connected to the Adm xxx50. Is this the correct library? (Y/N) Y
- The directory (with enough pre-requisite disk space) in which the extracted data will be placed. (A default directory is recommended.)
  Enter the root path for the extracted output files [/exlibris/aleph/u20_1/xxx50/scratch]:
- An optional email address to which a message email is sent when each area’s process is completed and a summary email is sent when all areas are completed.
  Enter your email address to receive notifications about the status of the extract processes:

Aleph jobs running in the background will slow the extract process and may stop it. Therefore, the extract reports if any Aleph jobs are running:

It is not recommended to run the extract while other scheduled jobs are running.

Searching for other scheduled jobs. Please wait...

Still searching ............

The extract reports currently running jobs and any jobs that are scheduled to run within 24 hours. If the extract detects running jobs, the following is displayed:

It is not recommended to run both the extract and the following jobs:

P_file_20

For more information view the following log:

/exlibris/aleph/a20_4/tmp/urm_heavy_jobs_32135_20150514_171449/urm_ps-ef_aleph_heavy_running.log

If the extract detects scheduled jobs, the following is displayed:

It is recommended to disable the following jobs that are scheduled within the next 24h:

p_manage_18

For more information view the following log: /exlibris/aleph/a20_4/tmp/urm_heavy_jobs_32135_20150514_171449/urm_sched_aleph_heavy_24h.log

You can exit the extract and review the logs or continue with the extract.

The data extract starts running after you answer the questions above. The data extract takes between two hours and two days to complete, depending on the amount of data and the server's hardware resources and performance. If the process is taking longer than this, contact your Ex Libris project manager.

After the process starts, you are prompted to a monitor screen where you can follow the progress of the data extract for each area. The screen refreshes every few seconds so you can see the progress:

Area Status Start End Extracted

========== ======= =========== =========== =========

09:37:15

ADAM Success 2013-OCT-02 2013-OCT-02 1

09:37:31 09:37:55

CALENDAR Success 2013-OCT-02 2013-OCT-02 208

09:37:31 09:37:45

CODE_TABLE Success 2013-OCT-02 2013-OCT-02 10

09:37:14 09:38:03

COURSE Process 2013-OCT-02 2000

09:37:16

E_RESOURCE Process 2013-OCT-02 20000

09:37:25

FUND Process 2013-OCT-02 0

09:37:14

FUND_TX Process 2013-OCT-02 0

09:37:15

INVENTORY Success 2013-OCT-02 2013-OCT-02 1860

09:29:48 09:37:13

INVOICE Pending 0

You can return to this screen at any time by opening the Utility menu and selecting option E.

In addition to the email completion updates, you can check the monitor status for each area, which should complete with Success. If any area completes with a status other than Success or Skipped, contact your Ex Libris project manager.

Note that the Invoice extract starts only after all other areas are extracted.

When all areas are extracted, the following message is displayed:

Extract has been Completed, Press Enter to Return to the Main Menu

Package and Transfer Extracted Files

When the extract is completed successfully, package, compress, and transfer it to Ex Libris.

To package, compress, and transfer the extracted directory:

Open the Utility menu and select option F. Package the Extracted Data.
Confirm the ADM Library:
You are connected to the Adm xxx50. Is this the correct library? (Y/N)
Select the data required for packaging. If the status of the migration is not fulfillment cutover, select option 1.
Please select the data required: 1. All areas 2. Fulfillment only (Patron/Request/Loan)
Enter the directory in which the extracted data is placed. This must be the same directory that you entered in step 3 of Setup and Data Extract (if you selected the default directory, use the default directory here as well by pressing Enter).
Enter the path of the location of the extracted data [/exlibris/aleph/u20_1/xxx50/scratch]: The process may take several minutes.
After the extract is completed, the extracted data is placed in the transfer directory. This directory is located as the subdirectory urm/extract/urm_xxx50/transfer under the path you entered above (the default location of the transfer directory is /exlibris/aleph/uXX_x/xxx50/scratch/urm/extract/urm_xxx50/transfer).
Open the Utility menu and select option G. Transfer the Packaged Data.
Confirm the ADM Library:
You are connected to the Adm xxx50. Is this the correct library? (Y/N) Y
Select the data required for packaging. If the status of the migration is not fulfillment cutover, select option 1.
Please select the data required: 1. All areas 2. Fulfillment only
Select the MFT server to which the data will be transferred, according to your region:
Select the mft server by the optional regions: 1.APAC 2.EUR 3.USA 4.CANADA 5.CHINA Please select your region:

If you have more than one Data Center (DC) in your region, you are asked to select the relevant DC sent to you by your Ex Libris representative.
If your region is USA, you are asked to select either the DC01 or the DC04 data center.Libris Secured File server

If your region is EUR, you are asked to select either the DC03 or the DC06 data center.
You are asked to enter your stfp user name and password. Use your SalesForce credentials. Copy and paste the key from the link sent to you by Ex Libris. Below is an example of the key you are asked to insert.

Enter your password:

-----BEGIN RSA PRIVATE KEY-----

MIIEoAIBAAKCAQEAz+h+RByvWo5aLNxpXS0hNh2DKlipYA81Ur+pVzoILXiA3EWW

h6zfCOxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxbP0UPtbKojfhIgCbAJrtwa7jszxu

XW9vn95ulDUNNuoyKGyyVeve7InsFx0NTzlLaJeJq3vfPCVXvO8InRY7giX/zCE0

yxsOal+Wsf5vn+9L8XGEdcaG37vPWGEBfaNYywIBIwKCAQBNORj0ujKAuIfkxuVO

fnoMySIXCv0VDPaMcxMDJC7sUVRvEoheR4YKnIVxq+edr12hLcdp7g5VjihxmxVP

-----END RSA PRIVATE KEY-----

If the connection cannot be established you will get the following message: Cannot establish connection to the mft server requested. Do you want to try again resetting your password? (Y/N) [N]
Enter the directory in which the extracted data will be placed. This must be the same directory that you entered in step 3 of Setup and Data Extract (if you selected the default directory, use the default directory here as well by pressing Enter).
Enter the path of the location of the extracted data [/exlibris/aleph/u20_1/xxx50/scratch]:
Enter your email address to receive notifications about the status of the transfer process.
Enter your email address to receive notifications about the status of the transfer process (* mandatory *):

After entering your email address, the data transfer begins.

Data is being transferred. You will be notified by e-mail when done.

The data is transferred securely using a secure file transfer protocol.
The data transfer process can take up to 48 hours to complete, depending on the size of your data and your Internet connection speed. The functioning of Aleph is not affected by this process running in the background, and your normal work in Aleph can proceed in parallel. If the process does not complete within 60 hours, contact Ex Libris.

Inform Ex Libris when the data transfer process is completed.

Fulfillment Cutover

The last stage of the data migration requires extracting fulfillment data (patrons , loans, and requests). The AutoExtract kit enables you to extract this data, package it, and transfer it. Do not download or install the kit now before fulfillment cutover. Use the kit and the Alma Migration form you installed for Cutover extract!

To extract the fulfillment area:

Move to the desired ADM library (dlib xxx50) and open the main menu by entering the following:
>> aleph2alma_menu
Select option D in the Aleph to Alma utility menu. The following is displayed:
Please select the data required:
1. All areas
2. Fulfillment only (Patron/Request/Loan)
Select option 2:
Only Patron/Request/Loan will be extracted. Correct? (Y/N)
Enter Y to start the fulfillment cutover (default is N). The Monitor screen appears.
After the extract completes, pack the data and transfer it as described in the sections above. For both packing and transferring the data, select the data required as Fulfillment only.

Extract Historical Loans

Select H Extract Historical Loans from the main menu to extract the data of the library's historical loans. It is recommended to perform the extract after the fulfillment cutover data extract and transfer are completed. The extract may take few hours, depending on the amount of data being extracted. Historical loans data can be found in the data_scratch of the extracted ADM library under /urm/extract/urm_xxx5n/loan_hist/loan_hist.csv.

Create a Test Migration Form and Run the Extract

During the Onboarding stage, you can create a test migration form to determine the following before generating the Alma Migration Form:

An estimation of how long the extract will take (relevant for a single institution run)
If the Aleph installation meets all of the requirements for extract

To create the test migration form, select option 3. Create a Dummy Migration Form.

There is no need to validate the test migration form. After you create it, run the extract as described in the Setup and Data Extract section.

This form contains the minimum required default values. These values cannot be used when running the real extract. If you attempt to run the extract with the test migration form, you are asked if you want to continue. Any related P2E steps are skipped. There is no need for the e_resource_input.csv file in order to run this test.

The data exported in a test extract is not useful, and cannot be loaded to Alma. Further, the test migration does not make any attempt to move data to the MFT. MFT access should be tested separately.

Troubleshooting

Cobol License Issue

If there is a problem when installing the kit (option A.) and the following message is displayed:

Error after install_aleph_2_urm

Refer to the installation log under TMPDIR:

$TMPDIR/a2u_install_sources*

Search for the following message:

Micro Focus LMF - 010: Unable to contact license manager. This product has been unable to contact the License Manager. Execution of this product has been terminated.

If the message above is displayed in the log, perform the following before reinstalling the kit:

Log to the server as the root user.
Run the following command:
>> /etc/mflmrcscript
Log on again as the aleph user and run option A.

Sometimes, even if the license is installed on the server, you may need to start the license server using /etc/mflmrcscript.

If the problem still occurs, contact your Ex Libris representative.

Oracle 12 Issue

When installing the kit, if the compilation of .ora or .pc files fails (compile_ora or compile_pc) with errors such as:

dbgc_init_all failed with ORA-48173

ORA-00600: internal error code, arguments: [17998], [2], [], [], [], [], [], [], [], [], [], []

ORA-00600: internal error code, arguments: [17998], [2], [], [], [], [], [], [], [], [], [], []

+ stack trace…

Preform the following workaround:

Log in as the oracle user.
rm -rf $ORACLE_BASE/diag/plsql
mkdir $ORACLE_BASE/diag/plsql
Reinstall the kit.

Copying the MFT SSH_KEY

In the extract process, you are asked to copy the ssh_key given to you by your Ex Libris representative. If you do not copy the entire key, the data is not packaged and transferred. If this situation occurs, do the following:

Close the extract menu.
Copy the entire content of the ssh_key (including the first and last "BEGIN"/"END" lines) and paste it into a text file in Windows.
Save the file with the following name: your generated MFT UserName + _urm_sftp_ssh_key (without any suffix). For example, if your MFT user name = MyGenUsrName, the ssh_key file is named: MyGenUsrName_urm_sftp_ssh_key.
Copy the ssh_key file to the Aleph server in the location where the Aleph2Alma kit was placed in the $alephm_proc directory.
Open a putty session to your Aleph server, and go to the $alephm_proc directory. Set permissions to your ssh_key file: chmod 600 MyGenUsrName_urm_sftp_ssh_key.
Rerun the extract menu. This time you are not asked to insert the ssh_key.