Data Module

The rocketlogger.data module provides support for processing binary RocketLogger data files.

RocketLogger Data Import Support.

File reading support for RocketLogger data (rld) files.

Copyright (c) 2016-2020, ETH Zurich, Computer Engineering Group All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

• Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

class rocketlogger.data.RocketLoggerData(filename=None, join_files=True, header_only=False, exclude_part=None, decimation_factor=1, recovery=False, memory_mapped=True)

Bases: object

RocketLogger data file handling class.

File reading and basic data processing support for binary RocketLogger data files.

The constructor takes the same parameters as load_file().

Parameters:

• filename – The filename of the file to import. If numbered files following the “<filename>_p#.rld” convention are found, they can be joined during import using the join_files parameter.

• join_files – Enable joining of multiple files, if numbered files following the “<filename>_p#.rld” convention are found

• exclude_part – Exclude given part(s) of number files when joining following the “<filename>_p#.rld” convention. Expects a single index or list or Numpy array of indexes to exclude. Silently ignores indexes beyond the number of found parts. Only applicable when joining multiple files with join_files=True.

• header_only – Enable to import only header info without data

• decimation_factor – Decimation factor for values read

• recovery – Attempt recovery of damaged files that have lost or incomplete data blocks at the end of the file. This can be caused by power supply failures during measurements.

• memory_mapped – Set False to fall back to read entire file to memory at once, instead of using memory mapped reading. Might increase file read performance for many smaller files and/or some system configurations.

add_channel(channel_info, channel_data)

Add a new data channel to the RocketLogger data structure.

Note

If a valid channel is to the channel being added, the linked channel has to be added first.

Parameters:

• channel_info –

  Channel info structure of the channel to add, a dictionary with the following fields designed:

  • ”unit_index” – index of the measurement unit of the channel

  • ”scale” – data unit scale, an integer exponent

  • ”data_size” – size in bytes of the channel data

  • ”valid_link” – index of the valid channel

  • ”name” – name of the channel (max 16 bytes)

• channel_data – The actual channel data to add, Numpy array

get_channel_names()

Get the names of all the channels loaded from file.

Returns:

List of channel names sorted by name

get_comment()

Get the comment stored in the file header.

Returns:

Comment stored in the file

get_data(channel_names=['all'])

Get the data of the specified channels, by default of all channels.

Parameters:

channel_names – The names of the channels for which the data shall be returned. List of channel names or “all” to select all channels.

Returns:

A Numpy array containing the channel’s data vectors

get_dataframe(channel_names=['all'], time_reference='relative')

Shortcut to get a pandas dataframe of selected channels indexed with timestamps.

By default all channels are exported and relative timestamps are used for indexing. See also get_data() and get_time() functions.

Requires pandas package to be installed.

Parameters:

• channel_names – The names of the channels for which the data shall be returned. List of channel names or “all” to select all channels.

• time_reference –

  The reference to use for timestamp calculation:

  • ”relative” – Calculate timestamp from sample rate, and the sample index relative to the measurement start time (default)

  • ”local” – Get the timestamp of the local oscillator clock

  • ”network” – Get the timestamp of the network synchronized clock

Returns:

A pandas dataframe the channel’s data as columns, indexed by the timestamps of the selected format

get_filename()

Get the filename of the loaded data file.

Returns:

The absolute filename of the loaded data file

get_header()

Get a dictionary with the header information.

The relevant fields include: “data_block_count”, “data_block_size”, “file_version”, “mac_address”, “sample_count”,”sample_rate”, and “start_time”.

To get the file comment use get_comment().

Returns:

Dictionary of relevant header information

get_time(time_reference='relative')

Get the timestamp of the data.

Using simple linear interpolation to generating the sample from the block timestamps. For local or network timestamps the values for the last data block are extrapolated using the average time delta of all data blocks.

Parameters:

time_reference –

The reference to use for timestamp calculation:

• ”relative” – Calculate timestamp from sample rate, and the sample index relative to the measurement start time (default)

• ”local” – Get the timestamp of the local oscillator clock

• ”network” – Get the timestamp of the network synchronized clock

Returns:

A Numpy array containing the timestamps

get_unit(channel_names=['all'])

Get the unit of the specified channels, by default of all channels.

Parameters:

channel_names – The names of the channels for which the unit shall be returned. List of channel names or “all” to select all channels.

Returns:

List of channel units sorted by channel_names list

get_validity(channel_names=['all'])

Get the validity of the specified channels, by default of all channels.

Parameters:

channel_names – The names of the channels for which the validity shall be returned. List of channel names or “all” to select all channels.

Returns:

A Numpy array containing the channel’s validity vectors

load_file(filename, join_files=True, exclude_part=None, header_only=False, decimation_factor=1, recovery=False, memory_mapped=True)

Read data from a RocketLogger data file.

Parameters:

• filename – The filename of the file to import. If numbered files following the “<filename>_p#.rld” convention are found, they can be joined during import using the join_files flag.

• join_files – Enable joining of multiple files if numbered files following the “<filename>_p#.rld” convention are found

• exclude_part – Exclude given part(s) of number files when joining following the “<filename>_p#.rld” convention. Expects a single index or list or Numpy array of indexes to exclude. Silently ignores indexes beyond the number of found parts. Only applicable when joining multiple files with join_files=True.

• header_only – Enable to import only header info without data

• decimation_factor – Decimation factor for values read

• recovery – Attempt recovery of damaged files that have lost or incomplete data blocks at the end of the file. This could e.g. be caused by power supply failures during measurements.

• memory_mapped – Set False to fall back to read entire file to memory at once, instead of using memory mapped reading. Might increase file read performance for many smaller files and/or some system configurations.

merge_channels(keep_channels=False)

Merge seamlessly switched current channels into a combined channel.

Parameters:

keep_channels – Whether the merged channels are kept

Returns:

Self reference to data object

plot(channel_names=['all'], show=True)

Plot the loaded RocketLogger data.

Requires matplotlib package to be installed.

Parameters:

• channel_names –

  The names of the channels for which the data shall be returned. List of channel names (or combination of):

  • ”all” – to select all channels

  • ”voltages” – to select voltage channels

  • ”currents” – to select current channels

  • ”digital” – to select digital channels

• show – Whether to show the plot window or not

Returns:

The matplotlib plot object used for plotting

remove_channel(channel_name)

Remove a data channel from the RocketLogger data structure.

Note

Linked valid channels are not automatically removed. If they shall be removed as well, call this function first on the linked channel.

Parameters:

channel_name – Name of the channel

exception rocketlogger.data.RocketLoggerDataError

Bases: Exception

RocketLogger data handling related errors.

exception rocketlogger.data.RocketLoggerDataWarning

Bases: Warning

RocketLogger data handling related warnings.

exception rocketlogger.data.RocketLoggerFileError

Bases: OSError

RocketLogger file read/write related errors.