Data
A data object contains your entire n-dimensional dataset, including axes, units, channels, and relevant metadata. Once you have a data object, all of the other capabilities of WrightTools are immediately open to you, including processing, fitting, and plotting tools.
Here we highlight some key features of the data object.
For a complete list of methods and attributes, see WrightTools.data.Data
in the API docs.
Instantiation
From Supported File Types
WrightTools aims to provide user-friendly ways of creating data directly from common spectroscopy file formats. Here are the formats currently supported.
name |
description |
API |
BrunoldrRaman |
Files from Brunold lab resonance raman measurements |
|
Cary |
Files from Varian’s Cary® Spectrometers |
|
COLORS |
Files from Control Lots Of Research in Spectroscopy |
|
JASCO |
Files from JASCO optical spectrometers |
|
KENT |
Files from “ps control” by Kent Meyer |
|
Aramis |
Horiba Aramis ngc binary files |
|
Ocean Optics |
.scope files from ocean optics spectrometers |
|
PyCMDS |
Files from PyCMDS |
|
Shimadzu |
Files from Shimadzu UV-VIS spectrophotometers |
|
SPCM |
Files from Becker & Hickl spcm software |
|
Solis |
Files from Andor Solis software |
|
Tensor 27 |
Files from Bruker Tensor 27 FT-IR |
Is your favorite format missing? It’s easy to add—promise! Check out Contributing.
These functions accept both local and remote (http/ftp) files as well as transparent compression (gz/bz2).
Compression detection is based on the file name, and file names for remote links are as appears in the link.
Many download links (such as those from osf.io or Google drive) do not include extensions in the download link,
and thus will cause Warnings/be unable to accept compressed files.
This can often be worked around by adding a variable to the end of the url such as https://osf.io/xxxxx/download?fname=file.csv.gz
.
Google Drive direct download links have the form https://drive.google.com/dc?id=XXXXXXXXXXXXXXXXXXXX
(i.e. replace open
in the “share” links with dc
).
From Bare Arrays
Got bare numpy arrays and dreaming of data? It is possible to create data objects directly, as shown below.
# import
import numpy as np
import WrightTools as wt
# generate arrays for example
def my_resonance(xi, yi, intensity=1, FWHM=500, x0=7000):
def single(arr, intensity=intensity, FWHM=FWHM, x0=x0):
return intensity*(0.5*FWHM)**2/((xi-x0)**2+(0.5*FWHM)**2)
return single(xi) * single(yi)
xi = np.linspace(6000, 8000, 75)[:, None]
yi = np.linspace(6000, 8000, 75)[None, :]
zi = my_resonance(xi, yi)
# package into data object
data = wt.Data(name='example')
data.create_variable(name='w1', units='wn', values=xi)
data.create_variable(name='w2', units='wn', values=yi)
data.create_channel(name='signal', values=zi)
data.transform('w1', 'w2')
Note that NumPy has functions for reading data arrays from text files. Our favorite is genfromtxt. Lean on these functions to read in data from unsuported file formats, then pass in the data as arrays. Of course, if you find yourself processing a lot of data from a particular file format, consider contributing a new from function to WrightTools.
Having trouble connecting the WrightTools Data structure to bare numpy arrays? We have a notebook that takes a look at how many common numpy.ndarray operations– slicing, element-wise math, broadcasting, etc.–have analogues within the WrightTools data structure:
Creating Compressed Datasets
WrightTools can transparently read and create compressed datasets by passing arguments to create_variable()
or create_channel()
.
These arguments are the same as are passed to h5py’s create_dataset method.
data = wt.Data(name='example')
data.create_variable(name='w1', units='wn', shape=(1024, 1024), compression="gzip")
data.create_channel(name='signal', shape=(1024, 1024), compression="gzip", compression_opts=9)
Structure & Attributes
So what is a data object anyway?
To put it simply, Data
is a collection of WrightTools.data.Axis
and WrightTools.data.Channel
objects.
WrightTools.data.Axis
objects are composed of WrightTools.data.Variable
objects.
attribute |
tuple of… |
|
|
|
|
|
|
|
As mentioned above, the axes and channels within data can be accessed within the data.axes
and data.channels
lists.
Data also supports natural naming, so axis and channel objects can be accessed directly according to their name.
The natural syntax is recommended, as it tends to result in more readable code.
>>> data.axis_expressions
('w1', 'w2')
>>> data.w2 == data.axes[1]
True
>>> data.channel_names
('signal', 'pyro1', 'pyro2', 'pyro3')
>>> data.pyro2 == data.channels[2]
True
The order of axes and channels is arbitrary.
However many methods within WrightTools operate on the zero-indexed channel by default.
For this reason, you can bring your favorite channel to zero-index using bring_to_front()
.
Variable
The WrightTools.data.Variable
class holds key coordinates of the data object.
One Variable
instance exists for each recorded independent variable.
This includes scanned optomechanical hardware, but also still hardware, and other variables like lab time.
A typical data object will have many variables (each a multidimensional array).
Variables have the following key attributes:
attribute |
description |
LaTeX-formatted label, appropriate for plotting |
|
variable maximum |
|
variable minimum |
|
variable name |
|
variable units |
Axis
The WrightTools.data.Axis
class defines the coordinates of a data object.
Each Axis
contains an expression
, which dictates its relationship with one or more variables.
Given 5 variables with names ['w1'
, 'w2'
, 'wm'
, 'd1'
, 'd2'
] , example valid expressions include 'w1'
, 'w1=wm'
, 'w1+w2'
, '2*w1'
, 'd1-d2'
, and 'wm-w1+w2'
.
WrightTools ignores the space character in expressions, so 'w1 = w2'
will be interpreted as 'w1=w2'
.
Axes behave like arrays: you can slice into them, view their shape, get a min and max etc.
But actually axes do not contain any new array information: they simply refer to the Variable arrays.
Axes have the following key attributes:
attribute |
description |
LaTeX-formatted label, appropriate for plotting |
|
coordinates minimum, in current units |
|
coordinates maximum, in current units |
|
axis name |
|
|
current axis units (change with |
component variables |
|
|
expression |
Constant
WrightTools.data.Constant
objects are a special subclass of Axis objects, which is expected to be a single value.
Constant adds the value to to the label attribute, suitable for titles of plots to identify
static values associated with the plot.
Note that there is nothing enforcing that the value is actually static: constants still have
shapes and can be indexed to get the underlying numpy array.
You can control how this label is generated using the attributes format_spec
an round_spec
.
label
uses the python builtin format
, an thus format_spec is a specification as in the
Format Specification Mini-Language.
Common examples would be “0.2f” or “0.3e” for decimal representation with two digits past the decimal
and engineers notation with 3 digits past the decimal, respectively.
round_spec
allows you to control the rounding of your number via the builtin round()
.
For instance, if you want a number rounded to the hundreds position, but represented as an integer, you may use round_spec=-2; format_spec="0.0f"
.
For example, if you have a constant with value 123.4567 nm
, a format_spec
of 0.3f
, and a round_spec
of 2
, you will get a label something like '$\\mathsf{\\lambda_{1}\\,=\\,123.460\\,nm}$'
, which will render as \(\mathsf{\lambda_{1}\,=\,123.460\,nm}\).
An example of using constants/constant labels for plotting can be found in the gallery: Custom Figure.
In addition to the above attributes, constants add:
attribute |
description |
|
Format specification for how to represent the value, as in |
|
Specify which digit to round to, as in round() |
LaTeX formatted label which includes a symbol and the constant value. |
|
The mean (ignoring NaNs) of the evaluated expression. |
|
The standard deviation of the points used to compute the value. |
Channel
The WrightTools.data.Channel
class contains the n-dimensional signals.
A single data object may contain multiple channels corresponding to different detectors or measurement schemes.
Channels have the following key attributes:
attribute |
description |
|
LaTeX-formatted label, appropriate for plotting |
channel magnitude (furthest deviation from null) |
|
channel maximum |
|
channel minimum |
|
channel name |
|
channel null (value of zero signal) |
|
flag to indicate if channel is signed |
Processing
Units aware & interpolation ready
Experiments are taken over all kinds of dynamic range, with all kinds of units.
You might wish to take the difference between a UV-VIS scan taken from 400 to 800 nm, 1 nm steps and a different scan taken from 1.75 to 2.00 eV, 1 meV steps.
This can be a huge pain!
Even if you converted them to the same unit system, you would still have to deal with the different absolute positions of the two coordinate arrays.
map_variable()
allows you to easily obtain a data object mapped onto a different set of coordinates.
WrightTools data objects know all about units, and they are able to use interpolation to map between different absolute coordinates. Here we list some of the capabilities that are enabled by this behavior.
method |
description |
gallery |
use interpolation to guess the value of NaNs within a channel |
||
join together multiple data objects, accounting for dimensionality and overlap |
||
re-map data coordinates |
Dimensionality without the cursing
Working with multidimensional data can be intimidating. What axis am I looking at again? Where am I in the other axis? Is this slice unusual, or do they all look like that?
WrightTools tries to make multi-dimensional data easy to work with. The following methods deal directly with dimensionality manipulation.
method |
description |
gallery |
chop data into a list of lower dimensional data |
||
destroy one dimension of data using a mathematical strategy |
||
destroy one dimension of a channel by taking the nth moment |
||
split data at a series of coordinates, without reducing dimensionality |
||
transform the data on to a new combination of variables as axes |
WrightTools seamlessly handles dimensionality throughout. Artists is one such place where dimensionality is addressed explicitly.
Processing without the pain
There are many common data processing operations in spectroscopy. WrightTools endeavors to make these operations easy. A selection of important methods follows.
method |
description |
gallery |
clip values outside of a given range (method of |
||
take the derivative along an axis |
||
join multiple data objects into one |
||
level the edge of data along a certain axis |
||
smooth a channel via convolution with a n-dimensional Kaiser window |