Data Input and Output (I/O)

The routines in this folder deal with data input and output. Transformers for converting list files to CRD files can also be found here. Also export functions are found here.

`CRDReader`

class rimseval.data_io.crd_reader.CRDReader(fname: Path)

Read CRD Files and make the data available.

Example:

>>> fname = Path("folder/my_crd_file.crd")
>>> crd_file = CRDReader(fname)
>>> crd_file.nof_ions
13281
>>> crd_file.nof_shots
5000

property all_data: Tuple[ndarray, ndarray]

Get the data.

Returns:: 1D array with ions per shot, 1D array with bin in which ions arrived

property all_tofs: ndarray

Get all time of flight bins.

Returns:: 1D array with bin in which ions arrived

property ions_per_shot: ndarray

Get ions per shot array.

Returns:: 1D array with ions per shot

property ions_to_tof_map: ndarray

Get the index mapper for ions_per_shot -> tof.

Returns:: Mapper with indexes where tofs are in all_tofs

property nof_ions: int

Get the number of shots.

Returns:: Number of shots.

property nof_shots: int

Get the number of ions.

Returns:: Number of ions.

parse_data(data: bytes) → None

Parse the actual data out and put into the appropriate array.

For this parsing to work, everything has to be just right, i.e., the number of shots have to be exactly defined and the data should have the right length. If not, this needs to throw a warning and move on to parse in a slower way.

Parameters:: data – Binary string of all the data according to CRD specification.
Warning:: Number of Shots do not agree with the number of shots in the list or certain ions are outside the binRange. Fallback to slower reading routine.
Warning:: There is more data in this file than indicated by the number of Shots.

parse_data_fallback(data: bytes) → None

Slow reading routine in case the CRD file is corrupt.

Here we don’t assume anything and just try to read the data into lists and append them. Sure, this is going to be slow, but better than no data at all.

Parameters:: data – Array of all the data.

read_data() → None

Read in the data and parse out the header.

The header information will be stored in the header dictionary. All entry names are as specified in the CRD format file for version 1.0.

Raises:

KeyError – Header is not available.
OSError – Corrupt data length.

CRD File Utilities

Header definitions, enums, and JIT compiled functions for processing CRD files.

Definitions

property rimseval.data_io.crd_utils.CURRENT_DEFAULTS Current default CRD header information.

property rimseval.data_io.crd_utils.HEADER_START Defaults sizes for the header values to read in.

`CRDHeader`

class rimseval.data_io.crd_utils.CRDHeader(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Enum class for CRD header.

The start must always be the same as HEADER_START above, however, the other fields might vary depending on the header that is being used. The header is called by its version number. Note that the letter v precedes the number and that the period is replaced with the letter p.

Format is always: Name, length, struct unpack qualifier

v1p0 = (('shotPattern', 4, '<I'), ('tofFormat', 4, '<I'), ('polarity', 4, '<I'), ('binLength', 4, '<I'), ('binStart', 4, '<I'), ('binEnd', 4, '<I'), ('xDim', 4, '<I'), ('yDim', 4, '<I'), ('shotsPerPixel', 4, '<I'), ('pixelPerScan', 4, '<I'), ('nofScans', 4, '<I'), ('nofShots', 4, '<I'), ('deltaT', 8, '<d'))

`shot_to_tof_mapper()`

rimseval.data_io.crd_utils.shot_to_tof_mapper(ions_per_shot: array) → array

Mapper for ions_to_shot to all_tofs.

Takes ions_per_shot array and creates a mapper that describes which ranges in the all_tofs array a given shot refers to.

Parameters:: ions_per_shot – Ion per shots array.
Returns:: Mappeing array shots to ToF.

LST to CRD File Transformer

class rimseval.data_io.lst_to_crd.LST2CRD(file_name: Path = None, channel_data: int = None, channel_tag: int = None)

Convert list files to CRD files.

Example:

>>> from pathlib import Path
>>> from rimseval.data_io import LST2CRD
>>> file = Path("path/to/file.lst")
>>> lst = LST2CRD(file_name=file, channel_data=1, tag_data=None)
>>> lst.read_list_file()
>>> lst.write_crd()

class ASCIIFormat(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Available formats for this routine that are already implemented.

Various formats that are implemented when dealing with ASCII data. The value is composed of a tuple of 2 entries.

0: entry: width of the binary number (binary_width)

1: Tuple of tuples with start, stop on where to read 0: sweep - 1: time - 2: channel

ASC_1A = (48, ((0, 16), (16, 44), (45, 48)))

ASC_9 = (64, ((1, 21), (21, 59), (60, 64)))

class BinWidthTDC(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bin width defined by instrument.

These are only FASTComTec instruments. The name of each entry is equal to the identifier that can be found in a datafile (lst). The entries of the enum are as following:

binwidth in ps

MCS8A = 80

MPA4A = 100

class DATFormat(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Available formats (time_patch) for binary data.

Various binary data formats are incorporated. Value is compmosed of 2 entries:

0: Data length in bytes

1: Encoding of the binary value to read with struct.unpack()

2: Tuple, Where in the decoded list are: 0: sweep - 1: time - 2: channel

DAT_9 = (8, '<')

property channel_data: int

Get / set the channel number of the data.

Returns:: Channel number of data
Raises:: TypeError – Channel number is not an integer.

property channel_tag: int

Get / set the channel number of the tag.

Returns:: Channel number of tag
Raises:: TypeError – Channel number is not an integer.

property data_format: ASCIIFormat

Select the data format to use to convert the LST file to CRD.

Returns:: The currently chosen data format.
Raises:: TypeError – Data format is not a DataFormat enum.

property file_name: Path

Get / set the file name for the file to be read / written.

Returns:: The path and file name to the selected object.
Raises:: TypeError – Path is not a pathlib.Path object.

read_list_file() → None

Read a list file specified in self.file_name.

This routine sets the following parameters of the class:

self._file_data
self._tag_data (if a tag was selected)

This routine sets the following information parameters in self._file_info:

“bin_width”: Sets the binwidth in ps, depending on the instrument
“calfact”: Calibration factor, to scale range to bins
“data_type”: Sets the data type, ‘ascii’ for ASCII or ‘dat’ for binary, str
“shot_range”: shot range
“timestamp”: Time and date of file recording
“time_patch”: Data format, as reported by Fastcomtec as time_patch. as str

Raises:

ValueError – File name not provided.
ValueError – Channel for data not provided.
OSError – The Data Format is not available / could not be found in file.
NotImplementedError – The current data format is not (yet) implemented.

set_data_format()

Set the data format according to what is saved in file_info dictionary.

The “data_type” and “time_patch” values must be present in the dictionary. Writes the information to itself, to the _data_format variable.

Raises:

NotImplementedError – Binary data are currently not supported.
ValueError – Needs to be binary or ASCII data.

write_crd() → None

Write CRD file(s) from the data that are in the class.

Note

A file must have been read first. Also, this routine doesn’t actually write the crd file itself, but it handles the tags, etc., and then sources the actual writing task out.

Raises:

ValueError – No data has been read in.
OSError – Data is empty.

LST File Utilities

JIT compiled functions for processing LST files.

`ascii_to_ndarray()`

rimseval.data_io.lst_utils.ascii_to_ndarray(data_list: List[str], fmt: ASCIIFormat, channel: int, tag: int = None) → Tuple[ndarray, ndarray, List]

Turn ASCII LST data to a numpy array.

Takes the whole data block and returns the data in a properly formatted numpy array. If channels other than the selected ones are available, these are written to a List and also returned as other_channels.

Parameters:

data_list – Data, directly supplied from the TDC block.
fmt – Format of the data
channel – Channel the data is in
tag – Channel the tag is in, or None if no tag

Returns:

Data, Tag Data, Other Channels available

`get_sweep_time_ascii()`

rimseval.data_io.lst_utils.get_sweep_time_ascii(data: str, sweep_b: Tuple[int, int], time_b: Tuple[int, int]) → Tuple[int, int]

Get sweep and time from a given ASCII string.

Parameters:

data – ASCII string
sweep_b – Boundaries of sweep
time_b – Boundaries of time

Returns:

sweep, time

`transfer_lst_to_crd_data()`

rimseval.data_io.lst_utils.transfer_lst_to_crd_data(data_in: ndarray, max_sweep: int, ion_range: int) → Tuple[ndarray, ndarray, bool]

Transfer lst file specific data to the crd format.

Parameters:

data_in – Array: One ion per line, two entries: sweep first (shot), then time
max_sweep – the maximum sweep that can be represented by data resolution
ion_range – Valid range of the data in multiples of 100ps bins

Returns:

Array of how many ions are in each shot, Array of all arrival times of these ions, and a bool if there are any ions out of range

Excel Writer

`workup_file_writer()`

rimseval.data_io.excel_writer.workup_file_writer(crd: CRDFileProcessor, fname: Path, timestamp: bool = False) → None

Write out an Excel workup file.

This is for the user to write out an excel workup file, which will already be filled with the integrals of the given CRD file.

Parameters:

crd – CRD file processor file to write out.
fname – File name for the file to write out to.
timestamp – Create a column for the time stamp? Defaults to False

Integral export / import

`export()`

rimseval.data_io.integrals.export(crd: CRDFileProcessor, fname: Path = None) → None

Export integrals to csv file.

If no file name is given, the file name of the CRD file is used, ‘_int’ is added, and ‘.csv’ is used as the extension.

Header lines start with a #, all data lines are comma separated and labels for rows and columns are given within the data.

Parameters:

crd – CRD file.
fname – File name to export to (optional): csv file.

Raises:

ValueError – CRD file has no integrals.

`load()`

Export functions

`tof_spectrum()`

rimseval.data_io.export.tof_spectrum(crd: CRDFileProcessor, fname: Path, bins: int = 1) → None

Export time of flight spectra to csv file.

Parameters:

crd – CRD file.
fname – File name to export to.
bins – How many data to bin.

`mass_spectrum()`

rimseval.data_io.export.mass_spectrum(crd: CRDFileProcessor, fname: Path, bins: int = 1) → None

Export time of flight and mass spectra to csv file.

Parameters:

crd – CRD file.
fname – File name to export to.
bins – How many data to bin.

Data Input and Output (I/O)

CRD File Utilities

Definitions

LST to CRD File Transformer

LST File Utilities

Excel Writer

Integral export / import

load()

Export functions

`load()`