Installing the NIST Crystal Data Search Software

The NIST Crystal Data Database contains limited information about >200,000 inorganic and organic crystalline materials. Typical entries contain unit cell and space group information, chemical formula and a literature reference. This database has not been updated in several years. The Database is available from the International Centre for Diffraction Data in a NIST-developed format, known as NBS*AIDS-83 (or NIST*AIDS-83). The database has also been distributed by NIST in a different format -- where information for each entry is partitioned into files -- each type of information is placed in a separate file. At present, NIST is not selling this database to the public in this latter format. Contact the Standard Reference Data Program, National Institute of Standards and Technology, 100 Bureau Dr., Stop 2310, Gaithersburg, MD 20899-2310, (301) 975-2008 (VOICE) / (301) 926-0416 (FAX), srdata@nist.gov (E-MAIL) for further information concerning obtaining this database.

The NIST Center for Neutron Research has developed software for accessing this database for our own research efforts. It requires the database in this latter, partitioned, format. Searches are performed over the web. While the software has been written in a fairly platform-independent manner, the details of compilation and implementation of web scripts varies between platforms. This document details the steps needed to install this software on a Linux web server that can be accessed only within NIST.

Software Distribution

The software is contained in file ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cdsearch.tar.gz and a set of demo data can be found in ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cdsearch_demo.tar.gz.

Software Compilation

Here are the steps involved in compiling the software. The examples are what I used for a Linux machine:
Download the software (say to /tmp/)
Create a directory to place the software
mkdir ~/crystaldata; cd ~/crystaldata
Unpack the software
tar xvzf /tmp/cdsearch.tar.gz
Modify file Make.inc
The definitions included here are:
FC
Defines the FORTRAN compiler (typically "f77" or "g77")
CC
Defines the C compiler (typically "cc" or "gcc")
F2CLIB
No longer used
DBG
may be ignored, unless debugging options are needed
NAMEOPTION
This option determines how the C routines are configured to match the usage in your particular FORTRAN compiler:
Use -DBIT16 if the C int type is 16 bits (INTEGER*2); use -DBIT32 if the C int type is 32 bits (INTEGER*4); use -DBIT64 if the C short (short int) type is 32 bits (INTEGER*4).
Use -DTWOSCORE if FORTRAN uses either 1 or 2 underscores for names (g77); use -DONESCORE if FORTRAN uses 1 underscore for names (SGI f77); use -DNOSCORE if FORTRAN uses no underscores for names (HP fort77).
See file src/cdfread.h to see how these values are used.
CFLAGS
Flags to be supplied to the C compiler -- this will depend on how your local system is configured.
FFLAGS
Flags to be supplied to the FORTRAN compiler -- this will depend on how your local system is configured.
Here is my example: 
  FC = g77 
  CC = gcc 
  F2CLIB =
# use next line for debugging only
#  DBG= -g -Wall

# options for C <-> Fortran naming
NAMEOPTION = -DTWOSCORE -DBIT32

CFLAGS = $(DBG) $(NAMEOPTION)
FFLAGS = $(DBG)
Install the database
Install either the demo version of the database, or the real database in subdirectory crystaldata (yes, that is crystaldata/crystaldata). I used a link:
mkdir demo
(cd demo; tar xvzf /tmp/cdsearch_demo.tar.gz)
ln -s demo crystaldata
Build & Run the ENCODE program
This program reads the distributed files and creates index files that are used for searching.
make encode-xterm
or
make encode

This creates 29 files in directory cdf (17 files named IND_cd*, 11 named *.bit and 1 named cd.sizes). If this does not work, you probably have a problem in complilation, check the Make.inc file and, if need be, consult a local programming guru.
Compile the search programs
make programs

Web Installation

There are four things that need to be accomplished here: 1) the Web pages used in searching must be placed in an area that can be read by the web server 2) a file must be created to invoke the cgi script 3) the cd.html file & cd-search.script files must be modified to access the cgi script & HTML files in the correct locations 4) the Crystal Data files must be protected so that users cannot read the files directly and thus pirate them. 5) misc customization
  1. I used
    cp -r html/* /var/www/html/xtal
    to put the web pages where they can be read.

  2. Create a file to execute the cd-search.cgi script 
    [root@charlotte cgi-bin]# ls -l cd-search.cgi 
-rw-r-x---    1 root     apache         51 Feb  4 16:01 cd-search.cgi
[root@charlotte cgi-bin]# more cd-search.cgi 
#!/bin/sh
ulimit -c 0 -t 10 -f 500
/home/toby/crystaldata/cgi/cd-search.cgi

  3. I changed the "FORM METHOD=POST ACTION=" line to reference "/cgi-bin/cd-search.cgi" so that file /var/www/cgi-bin/cd-search.cgi would be used in both cd.html and cd-search.script.

  4. I used a variety of commands to make ~toby/crystaldata accessible to the web server but not the general user. Note that the web server runs as UID=apache, GID=apache 
    chmod o+x ~toby
chmod o+x ~toby/crystaldata
chmod o+x ~toby/crystaldata/cgi
cd ~toby/crystaldata
chmod -R o-r database/ demo/ cdf/
chmod o-x database/ demo/ cdf/
chgrp -R apache database/ demo/ cdf/
chmod -R g+r,g-w,g-x database/ demo/ cdf/
chmod g+x database/ demo/ cdf/

  5. Customization/Fixes
    • Other: fix reference to background image Change reference to image file: see line with Background="/xtal/images/bkground.jpg" in cgi/cd-search.cgi
    • Change the line in cd-search.script [look for variable conf(inetlist)] that determines the web addresses allowed to access the database. You may also need to change conf(helpfile) and conf(mainpage) to point to the URL's for these files.
    • Note the other parameters near the top of this file.

Test with demo Database

Search for Si -- all elements allowed (197 hits are obtained). Browse the list. Click on an entry in the browse list.

Install the Full Database

cd ~/crystaldata
rm crystaldata
ln -s database crystaldata
make export-xterm
rm /tmp/cdsea/*
The software described here was written by Brian Toby, NIST Center for Neutron Research, with help from Vicky Lynn Karen, Ceramic Division. This software is not subject to copyright -- it may be modified and distributed by anyone. (Please send additions and corrections to Brian Toby so that others may also benefit from your efforts.) Note that the NIST Crystal Data Database is subject to copyright and may not be freely distributed; NIST is required by law to recover costs incurred in the production of this database.

This software is supplied "as is" and without support. It is the policy of the NCNR to provide our internally-developed software to the scientific community at large and to our facility's users, specifically. However, we cannot promise support if the software fails to work properly. Neither the U.S. Government nor the authors make any warranty, expressed or implied, or assume any liability or responsibility for the use of this software. Brand names cited here are used for identification purposes and do not consitute an endorsement by NIST.

Brian Toby (Brian.Toby@NIST.GOV)
$Revision: $ $Date: $