SANS Data Treatment

Version 1.992

R. Ghosh, July 2023


NEWS


Programs may be found on the ftp server at the ILL ftp://ftp.ill.fr/pub/cs/sans, or at http://hookds.synology.me/softron

The April 2018 update to Windows-10 included changes to font metrics for full-HD displays and the graphics symbol characters were no longer compatible with GRWND.EXE. The pgplot library and programs were modified to generate vector drawn symbols in June; the corresponding plot files are now slightly larger but give correct representations.

NeXus to standard ascii transcription.
A set of programs has been written to examine and rewrite NeXus data files as standard ascii files offering compatibility with the existing programs. A sequence can be transformed rapidly into equivalent ascii data files. Program versions for Linux, Macintosh OS-X and Windows are available.

WINDOWS-7 download and unzip files onto a USB-stick
Files downloaded from the internet are marked and must be individually unlocked before programs can be executed. To obviate these protections, if the files are downloaded (or recopied) onto a USB-stick (which is usually formatted as a FAT32 file-system) and unzipped there, the resultant files may then be recopied back onto the hard disk without this download signature.

To account for new systems (Windows-7/8, Mac-OSX updates) a number of changes have been introduced. ftp://ftp.ill.fr/pub/cs/sans Here will be found distributions for PC and Mac-OSX. The PC version includes Prop to aid running programs in the PC-Windows environment. For Mac-OSX a startup file sassetup is included as is found on the linux distribution and also described in the manual. In these distributions there is additional documentation on improvements; this include caching uncompressed data examined using xplots and plug-in programs allowing data from different sources to be plotted and fitted.

In parallel to the standard regrouped data output, XML formatted files are now also produced. These conform to the schema definitions of the canSAS working party. A style sheet xg.xsl is also copied to the working directory. This allows the xml files to be "dropped" onto browsers such as IE6, or firefox where the tabulated data can be viewed. Dropping the data onto Excel and invoking this style sheet when asked brings the data into a worksheet for treatment.

2004 An updated set of programs for treating ILL SANS data has been prepared for PCs. A program launcher PROP for WindowsXP etc greatly simplifies program use, and the X-window program versions can now also function similarly in Windows.

The current SANS Computing Guide can by obtained here: sans_manual.pdf
This file is about 2.4Mb (143 pages), and can be viewed and printed using the Adobe Acrobat file reader.


Introduction

Using the Unix system

Raw data programs

Regrouping Programs

Data Corrections

Data Presentation

Anisotropic Treatments

Interactive Analysis

SANS programs available for PCs

Reading in data from other facilities

Tools


Introduction

At the ILL there is both an advantage and a disadvantage in the rate at which SANS measurements can be made. Full surveys over a range of controlled parameters can be made, but the number of runs for subsequent treatment can be large. Often only a sub-set are analysed during the experiment. To simplify full analysis the following programs are available not only at ILL ftp://ftp.ill.fr/pub/cs/sans, but are at http://hookds.sysnology.me/softron to run on PCs, Macs or Linux. Part treated or raw data can be exported on USB drives or CDroms.

The present page is intended to survey the programs available at ILL for treating SANS data. Much of the success of experiments depends on minimising background effects, and achieving high reproducibility in performing measurements. The simple procedures for treating data still require carefully measured transmissions to enable the various component spectra to be combined to a valid result.

During various stages of treatment the components are in files with compatible formats, and it is hence possible to use the standard tools shown below to identify problems arising from an occasional mis-measurement of a single component. Each data file contains information on its source and it is hence easy to trace errors arising from incorrectly measured or analysed constituents.

On all SANS instuments the data follow the ILL standard specification The raw data from D11, D17, and D22 are similarly formatted, but D16 retains the standard diffractometer format. When looking at images of the detector the standard view for D11, D17 and D22 is in cartesian coordinates looking from the sample. For D16 the view is towards the sample.

All raw data are accessed by routines which attempt to cope with the different raw data formats. Most programs can be used to treat data from D11, D16, D17 or D22.

The LAMP package based on IDL can be used to examine raw data from the multi-detectors, and perform some regrouping tasks.

Treated data are written as ASCII (plain text) files.

Radially regrouped data are stored in files with names:

		gnnnnnn.mmm,    e.g. g006123.001
with an identifying run number nnnnnn, and an extension number 1.

Two dimensional data are stored in files with names:

                tnnnnnn.mmm	e.g. t011021.001

Most programs which treat two dimensional data are capable of reading either raw or treated data.

Instrument identification and location of raw data

Unix environment variables are used to define the location of the data, and the name of the current instrument. These are normally predefined for the visitor file-systems.

e.g.         % setenv SAS_DATA_PATH  /usr/illdata/data/d11/
 identifies the directory /usr/illdata/data/d11/ as the raw data source
 for the data from the current cycle

             % setenv SAS_INSTRUMENT d11
 identifies D11 as the current instrument

Other environment variables which are used by the plotting programs include:

PGPLOT_DIR=../libsLgf
PGPLOT_ILL_DEV_1=/xserv
PGPLOT_ILL_DEV_2=/vcps
PGPLOT_ILL_PGRUNFIGDISP=figdisp
PGPLOT_ILL_PPAGE=2
PGPLOT_XW_WIDTH=0.55
All plotting expects the individual pictures to have a width/height aspect ratio of about 4:3. With settings as above the resulting x-y graphs are drawn two per A4 page in portrait format. If only one picture per page is required then plots must be made in landscape mode with:
PGPLOT_ILL_DEV_2=/cps
and
PGPLOT_ILL_PPPAGE=1

return to index

Using the Unix system

The environment on the instrument visitor zones has been setup to offer each user their own file-space. Users arriving fresh to the instrument might find it convenient to log-off and relog-on with their own names to localise their result files.

Users of other login names and those working on other workstations should note that they can set up their environment with the following commands (which might eventually be added to their .cshrc files)

	% setenv SAS_DIR ../sasprogdir   ;distribution directory
	% source $SAS_DIR/sassetup
If the workstation is not setup with a standard configuration this might fail and the following alternative might work at the ILL:
        % setenv SAS_DIR /hosts/serhom/home/cs/sans
        % source $SAS_DIR/sassetup

Programs are launched by typing their names after the shell (system) prompt.

The program name is used to invoke its operation.

Some useful commands when typing into a terminal window are

% ls				directory command
	e.g.% ls g0*		shows all regrouped data files
				g0nnnnn.nnn
% cat				types a file
	e.g.% cat g008700.001	types out regrouped data file
				for 8700 extension 1

% more				types a file in pages
	e.g.% more rnls57.lis
% pri				prints a file
	e.g.% pri pgplot58.ps
return to index

Raw data programs

dirrun

This program gives a summary of the raw data files for a particular instrument.

   e.g.
 % dirrun d11
 Data in directory /usr/illdata/data/d11                             
 numors 004362 to 006284
 numors 006286 to 008435
 4940 data files are stored here
If a full directory pathname is given then this is used instead of the instrument's current default directory /usr/illdata/data/instrument/

rundex

This program gives a summary extracted from either raw, 1D or 2D treated data showing monitors, wavelength etc associated with each data file. For the raw data there are additional options to select and print parameters from these files in tabular form.

detec, detd16

This produces a listing of the data; the user can select various parts of the multi-detector matrix. The smaller detector of D16 can be examined more simply using detd16.

windet

This produces a screen listing of a selected range of the datector data, with options to produce a listing file. If a reference transmssion run has been defined then ratio of the sum of the windows is printed. This is intended to aid assessment of transmission meaurements. To help assess the beam centre the cofg and the midpoint at half-height of the displayed region is calculated.

splots and xplots

These programs show coloured pixel maps of the raw data, with the possibility of creating PostScript plots with one, or, allowing comparisons, up to 6 pictures per page. The xplots version only runs on colour-graphics X-terminals, but incorporates a button- pushing interface. xplotc includes a cursor function on the pixel map.

anspec

This draws isometric projections of the multi-detector data on screen and for PostScript hardcopy.

grund

This program is used to examine a nominally flat spectrum, and, interactively, allows cell contents outside fixed limits to be identified, and a mask created to act as a filter in subsequent analysis.

cont

Contour plots of raw or treated data are produced interactively.

colrow

This program sums, plots and lists multidetector data, locating the maximum, and estimating peak widths. Data can be normalised with a nominally flat spectrum for removing efficiency and solid angle effects.

rmask

This can generate mask patterns with different geometries which then can be applied to eliminate bad data before treatment.

return to index

Regrouping Programs

These programs, and the programs which display and analyse these regrouped data are Fortran programs which have been compiled with a limit of 180 Q,I,errI values per spectrum. The limit is a consequence of fixed storage imposed by the language, and the general access routine RGIO has the limit set in a block data statement, which can thus be overridden without modifying the rlib library.

rnils

Performs basic regrouping, presenting the results of sequences of experiments side by side, simplifying comparisons.

snils

Performs regrouping, but iterates to eliminate cells which are intermittently noisy, which cannot easily be masked out using grund or rmask.

areg

Regroups data with the possibility of defining multiple segments.

iqpsi

Regroups data radially in large numbers of sectors, plotting results. Results can be saved in a text file.

ananas

Regroups data as a ribbon of boxes at an angle which may be set to match a 1D ordered sample axis.

rnd16 and rnd16s

A version of rnils for D16, the latter incoporating detector efficiency corrections.

red16

Treats one or a set of two-theta scans on D16, and regroups data with efficiency and solid angle corrections if required. Results may be stored in a format compatible with other SANS programs.

To obtain a summary and index of data files use rundex

return to index

Data Corrections

spolly, xpolly, mpolly, anpoly, mill, xmill

These programs combine regrouped and unregrouped data respectively, and create resultant files where the extension number is uniquely incremented. xpolly is a version of spolly with a button- pushing interface, and extra features. mpolly can be used for sequences of sample measurements when other correction factors remain constant.

Simple corrections dealing with a sequence of samples can be performed with mill and xmill. These treat either 1D or 2D data in the same manner. The second provides push-button controls.

return to index

Data Presentation

In general, sets of similar samples are compared, hence comparative display plays an important part in analysis of results.

xplots, colrow, and anspec will all display treated 2D data

rplot

This plots a number of functions of intensity and scattering vector for up to two sets of regrouped data.

mplot, xmplot

Plots up to 10 regrouped spectra. A number of functional forms may be selected. The second has pushbutton controls, and allows printing without leaving the program.

rlist

Lists regrouped data in a similar fashion to SPOLLY, thus the table includes "radius", theta, Q, Q**2, Q**-4, I, dI, IQ**2 etc.

smorger

A utility program which merges regrouped data, either by concatenation or sorting into increasing magnitude of Q. The program will also trim points off the ends of data to be incorporated.

To obtain a summary and index of files use rundex

return to index

Anisotropic Treatments

ansmoo

Performs local smoothing of data

ancos2

Analyses data as elliptical distortions typically due to external constraints.

To obtain a summary and index of files use rundex

return to index

Interactive Analysis

rguim

This fits straight lines to selected regions of data points, deriving a "radius of gyration" for a number of simple structural models.

sasgau, saspks

Uses the FITFUN library to fit gaussian and other peaks in regrouped data

sasfit4

Use the FITFUN library to fit gaussian and other peaks in regrouped data for sequences of data. The starting parameters are refined within a FITFUN environment. Then sequences of data may be fitted with parameters being systematically updated, or the original starting vaulues being re-used for each set. This allows overlapping peaks which change intensity and/or position to be analysed quite sensitively. Output is given suitable for spreadsheet use.

ssqfun

Uses the FITFUN library to fit concentric shell models to regrouped data.

polymer

Uses the FITFUN library to fit selected polymer models to regrouped data (A R Rennie, Uppsala, Sweden).

sqcon

Uses the FITFUN library and routines by Hayter/Penfold to fit SANS patterns from concentrated macroions in solution (A R Rennie, King's College, London).

A number of other programs use the FITFUN library to treat SANS data.

return to index

Reading Data from other facilities

A number of programs have been written to transform data from other centres to enable further treatment by ILL programs. These include:

ralread		reads data from LOQ at ISIS, Rutherford Lab.
esrfrill	reads raw data from ESRF SAS
esrfgill	reads regrouped data from ESRF SAS
postcard	reads old ILL RCARD data
colin		reads multi-column spreadsheet data (skips top)
		and stores in ILL format for standard use

Self-Service Tools

librlib.a

This library contains routines to access raw and treated data in a systematic fashion, with reasonable error reporting, and some debugging facilities built-in. This has been used extensively in the above programs. Plotting routines are also included.

libpgplot.a

This contains the PGPLOT plotting package written and maintained by Dr. T.J. Pearson of the California Institute of Technology, with some extensions developed at ILL including multiple open devices.