API Reference — RF Tools¶

embers.rf_tools is used to pre process, condition and preview raw rf data.

It contains rf_data, align_data, colormaps modules.

embers.rf_tools.rf_data.read_data(rf_file=None)¶

Convert rf binary data into ndarray of power and time.

from embers.rf_tools.rf_data import read_data
power, times = read_data(rf_file='~/embers-data/rf.txt')

Parameters:	rf_file – path to rf binary data file str
Returns:	power - power in dBm ndarray times - times in UNIX ndarray

embers.rf_tools.rf_data.tile_names()¶

List of MWA and reference antenna names

from embers.rf_tools.rf_data import tile_names
tiles = tile_names()

print(tiles)
>>> ["rf0XX", "rf0YY", "rf1XX", "rf1YY", "S06XX", ....]

Returns:	tiles - antenna names list

embers.rf_tools.rf_data.tile_pairs(tiles)¶

Create a list of all possible AUT ref antenna pairs from tile_names().

Reference and MWA antennas can only pair with antennas of the same polarizarion

from embers.rf_tools.rf_data import tile_names, tile_pairs
tiles = tile_names()
tile_pairs = tile_names(tiles)

print(tile_pairs)
>>> [["rf0XX", "S06XX"], ["rf0YY", "S06YY"], ....]

Returns:	pairs - all possible antenna pairs of same polarization list

embers.rf_tools.rf_data.time_tree(start_date, stop_date)¶

Split a date interval into 30 min observation chunks.

This is used to travers the following directory tree. The data_root dir contains a sub-dir for every tile in tile_names() within which are dirs for every day, containg raw rf data files recorded every 30 minutes

data_root
├── tile1
│   └── YYYY-MM-DD
│       └── tile1_YYYY-MM-DD-HH:MM.txt
└── tile2
    └── YYYY-MM-DD
        └── tile2_YYYY-MM-DD-HH:MM.txt

from embers.rf_tools.rf_data import time_tree
dates, time_stamps = time_tree("2020-01-01", "2020-01-02")

print(dates)
>>> ["2020-01-01", "2020-01-02"]
print(time_stamps)
>>> [["2020-01-01-00:00, 2020-01-01-00:30", ...],
        ["2020-01-02-00:00", "2020-01-02-00:30"]]

Parameters:	start_date – in YYYY-MM-DD format str stop_date – in YYYY-MM-DD format str
Returns:	A tuple (dates, time_stamps) dates - list of days between start_date and stop_date list time_stamps - list of list with all 30 minute observation in each day list

embers.rf_tools.rf_data.plt_waterfall(power, times, name)¶

Create waterfall plot() object from rf data.

waterfall created using parameters power, times from read_data(). Default unix times are converted to a human readable HH:MM format.

Parameters:	power – ndarray object from read_data() times – ndarray object from read_data() name – name of rf data identifier to add to plot title
Returns:	plt - plot() object

embers.rf_tools.rf_data.single_waterfall(rf_file, out_dir)¶

Save a waterfall plot from rf data file.

Parameters:	rf_file – path to a rf data file str out_dir – path to output directory str
Returns:	waterfall plot saved by savefig()

embers.rf_tools.rf_data.batch_waterfall(tile, time_stamp, data_dir, out_dir)¶

Save a waterfall plot for a batch of rf data files.

Parameters:	tile – tile name str time_stamp – start of rf observation in YYYY-MM-DD-HH:MM format str data_dir – path to root of data directory str out_dir – path to output directory str
Returns:	waterfall plot saved by savefig()
Raises:	FileNotFoundError – input file does not exist

embers.rf_tools.rf_data.waterfall_batch(start_date, stop_date, data_dir, out_dir)¶

Save a series of waterfall plots in parallel.

Parameters:	start_date (str) – date in style YYYY-MM-DD stop_date (str) – date in style YYYY-MM-DD data_dir (str) – path to root of rf data dir out_dir (str) – path to output dir

embers.rf_tools.align_data.savgol_interp(ref, tile, savgol_window_1=None, savgol_window_2=None, polyorder=None, interp_type=None, interp_freq=None)¶

Interpolate a power array followed by savgol smoothing.

Interpolate to a given frequency, making the dimensions of the power arrays from reference and tile antennas equal, enabling comparisons between corresponding data points. Two level of savgol filter applied, first to capture deep nulls + small structure, and second level to smooth over noise.

from embers.rf_tools.align_data import savgol_interp

 sg_interp_tuple = savgol_interp(
                    "~/embers-data/rf0XX.txt",
                    "~/embers-data/S06XX",
                    savgol_window_1=11,
                    savgol_window_2=15,
                    polyorder=2,
                    interp_type="cubic",
                    interp_freq=1)

(ref_ali, tile_ali, time_array,
    ref_power, tile_power, ref_time, tile_time) = sg_interp_tuple

Parameters:	ref – path to reference data file str tile – path to tile data file str savgol_window_1 – window size of savgol filer, must be odd int savgol_window_2 – window size of savgol filer, must be odd int polyorder – polynomial order to fit to savgol_window int interp_type – type of interpolation. Ex: ‘cubic’, ‘linear’ str interp_freq – freqency to which power array is interpolated in Hertz int
Returns:	A tuple (ref_ali, tile_ali, time_array, ref_power, tile_power, ref_time, tile_time) ref_ali - aligned reference power array tile_ali - aligned tile power array time_array - corresponding to power arrays ref_power - raw reference power array tile_power - raw tile power array ref_time - raw reference time array tile_time - raw tile time array

embers.rf_tools.align_data.plot_savgol_interp(ref=None, tile=None, savgol_window_1=None, savgol_window_2=None, polyorder=None, interp_type=None, interp_freq=None, channel=None, out_dir=None)¶

Plot single channel of power arrays to visualise savgol_interp().

Create a plot of a single channel of raw rf_data from reference and tile power arrays, along with the outputs of savgol_interp() to visualise the effects of interpolation and savgol smoothing.

Parameters:

ref – path to reference data file str tile – path to tile data file str savgol_window_1 – window size of savgol filer, must be odd int savgol_window_2 – window size of savgol filer, must be odd int polyorder – polynomial order to fit to savgol_window int interp_type – type of interpolation. Ex: ‘cubic’, ‘linear’ str interp_freq – freqency to which power array is interpolated in Hertz int channel – index of single frequency channel int out_dir – path to output directory str

Returns:

single freqency savgol_interp plot saved to out_dir

embers.rf_tools.align_data.save_aligned(tile_pair, time_stamp, savgol_window_1, savgol_window_2, polyorder, interp_type, interp_freq, data_dir, out_dir)¶

Save an aligned set of rf data with savez_compressed() to an npz file.

A pair of rf data files are smoothed, interpolated and aligned with the savgol_interp(). with the output written to a npz file and saved to an output directory tree.

from embers.rf_tools.align_data import save_aligned

savgol_interp(
            ["rf0XX", "S06XX"],
            "2020-01-01-00:00"
            savgol_window_1=11,
            savgol_window_2=15,
            polyorder=2,
            interp_type="cubic",
            interp_freq=1,
            "~/embers-data/",
            "~/embers-outputs")

Parameters:	tile_pair – pair of ref and tile antenna names from tile_pairs() list time_stamp – time when rf observation began. In YYYY-MM-DD-HH-MM format str savgol_window_1 – window size of savgol filer, must be odd int savgol_window_2 – window size of savgol filer, must be odd int polyorder – polynomial order to fit to savgol_window int interp_type – type of interpolation. Ex: ‘cubic’, ‘linear’ str interp_freq – freqency to which power array is interpolated int data_dir – root of data dir where rf data is located str out_dir – relative path to output directory str
Returns:	aligned rf data saved to npz file by savez_compressed()
Raises:	FileNotFoundError – an input file does not exist

embers.rf_tools.align_data.align_batch(start_date=None, stop_date=None, savgol_window_1=None, savgol_window_2=None, polyorder=None, interp_type=None, interp_freq=None, data_dir=None, out_dir=None, max_cores=None)¶

Temporally align all RF files within a date interval using save_aligned().

Parameters:

start_date – In YYYY-MM-DD format str stop_date – In YYYY-MM-DD format str savgol_window_1 – window size of savgol filer, must be odd int savgol_window_2 – window size of savgol filer, must be odd int polyorder – polynomial order to fit to savgol_window int interp_type – type of interpolation. Ex: ‘cubic’, ‘linear’ str interp_freq – freqency to which power array is interpolated int data_dir – root of data dir where rf data is located str out_dir – relative path to output directory str max_cores – Maximum number of cores to be used by this script. Default=None, which means that all available cores are used

Returns:

aligned rf data saved to npz file by savez_compressed() in out_dir

embers.rf_tools.colormaps.spectral()¶

Beautiful non-linear spectral colormap

spectral() is not perceptually uniform and is only used to easily preview raw data with high contrast

Returns:	spectral, spectral_r - ember colormap ListedColormap.

embers.rf_tools.colormaps.jade()¶

Beautiful perceptually uniform jade green colormap.

Returns:	jade, jade_r - ember colormap ListedColormap

embers.rf_tools.colormaps.waves_2d()¶

Create 2d sine wave.

Returns:	sine2d- 2d sine wave ndarray

embers.rf_tools.colormaps.plt_colormaps(spec, spec_r, jade, jade_r, out_dir)¶

Plot 2x2 grid of sample colormaps.

Parameters:	ember_cmaps – custom ember colormaps ListedColormap out_dir – path to output directory str
Returns:	waterfall plot saved by savefig() to out_dir