The Tools

The Saxsdog

The saxsdog is a script that converts directories with images to curves. It can use multiple threads and watch the file system for changes.

For help on the usage type:

$ saxsdog --help
Usage: saxsdog [options] directory/to/watch

Options:
  -h, --help            Show this help message and exit.
  -c FILE, --calibration=FILE
                        Path to calibration file (JSON).
  -t THREADS, --threads=THREADS
                        Number of concurrent threads.
  -m, --plotmonitor     Show a live updating plot window.
  -w, --watch           Watch directory for changes, using file system events
                        recursively for all sub directories.
  -r, --resume          Skip files that are already converted.
  -o OUTDIR, --out=OUTDIR
                        Specify output directory.
  -R RELPATH, --relpath=RELPATH
                        Specify output directory as relative path to image
                        file. Default: '../work'
  -s, --svg             Write plot to svg file.
  -p, --png             Write png of original.
  -P, --profile         Make a time Profile and print it.
  -S, --silent          Less output.
  -n, --nowalk          Don't scan for files already there, only watch file
                        system if -w flag is given.
  -D, --walkdirinthreads
                        Search all directories in parallel process.
  -V, --servermode      Servermode.

The calibration file must be a valid Calibration file Reference

The Converter

The converter extracts information from the calibration.txt generated by fit2d and adds them to a SAXS.calibration configuration file. (Calibration file Reference)

$ saxsconverter  --help
Usage: 
To extract data from Fit2d output:
  saxsconverter [options] calibration.txt ouput.saxsconf
Or to convert fom older saxsconf:
  saxsconverter [options] cal.saxsconf ouput.saxsconf


Options:
  -h, --help            show this help message and exit
  -t FILE, --template=FILE
                        Path to calibration file which serves as template.

If there is a target file and it is a valid Calibration file Reference, the parsed values are added or updated in place.

Plotchi

The tool “plotchi” plots a list of ”.chi”-files:

$ plotchi  --help
Usage: plotchi [options] CHIFILE [List of more ".chi" files]

Options:
  -h, --help            show this help message and exit
  -o FILE, --out=FILE   Write the plot to FILE. The format is derived from the
                        suffix, e.g. '.svg','.pdf'.
  -c, --compare         Compare datasets to first one.
  -l, --log             Use log scale.
  -n, --no-legend       Hide legend.
  -t TITLE, --title=TITLE
                        Give plot title.
  -s N, --skip=N        Skip first N points.
  -k N, --clip=N        Clip last N points.
  -x TYPE, --xaxsistype=TYPE
                        Select type of X axis scale, might be
                        [linear|log|symlog]
  -y TYPE, --yaxsistype=TYPE
                        Select type of Y axis scale, might be
                        [linear|log|symlog]

Saxsdmerge

This tool adresses a very specific problem only applicable under our beamlines needs.

$ saxsdmerge   --help
Usage: saxsdmerge [options] iMPicture/dir peakinteg.log datalogger.log

Options:
  -h, --help            show this help message and exit
  -t SEC, --timeoffset=SEC
                        Time offset between logging time and time in
                        imagedata.
  -1, --syncfirst       Sync time by taking the time difference between first
                        shutter action and first image.
  -o FILE, --outfile=FILE
                        Write merged dataset to this file. Format is derived
                        from the extesion.(.csv|.json|.hdf)
  -b, --batch           Batch mode (no plot).
  -c, --includechi      Include radial intensity data (.chi) in hdf.
  -f, --includetif      Include  all image data in hdf.
  -C FILE, --conf=FILE  Use config in  FILE to merge the data (ignore other
                        options)

What it does is the following: It merges the two log files by interpolating the missing data in the joined table of the the 1 sec interval data logger and the shutter log. The interpolation method takes the closest previous entry. The two logfiles are presumed to have the same clock. This data is then merged with the data extracted from the image headers and dedector logs and is written to a output file as a Table. As the image time stamps are from a different clock which might have a significant offset, the -1 and -t options allow for dealing with that. In order to check if the time synchronization is reasonable, the tool displays a graph with the shutter times and the exposure times from the images.

One of the output file format options is hdf, the hirarchical data format, which ia a way to create portable binary data files. If you specify hdf, the radial average function “.chi” and even the image data can be written to the hdf file. The hdf export is done with pandas hdf exporter which in turn uses PyTables. The “.chi” files reside under the “Graphs” Label, the log data under the “LogData” label and the images if present under the “Images” Label. It can be restored to a Pandas pannel or data frame with:

import pandas as pd
store=pd.HDFStore("/home/chm/saxs/merged.hdf")
Curves=store["Curves"] # gives Pandas panel with all the curves
Data=store["Data"] # gives Pandas data frame of merged log and image parameters