module: utils

Collection of practical utils, such as deterministic and stochastic signals generators and a collection of recorded biological signals, see Signal Catalog. This utils are intended to accelerate algorithm development and/or for educative purposes.

Signal Generators

Class of (test) signal generators for often used signals.

Introductory Example

Example demonstrates use of signal generators in a explanatory (longer) or a compact (shorter) version.

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

# -------- Explanatory (longer) version ----------
K = 500  # number of sample of the signal
N = 5  # number of unit impulses
K_sin = 20  # number of sample of the sinusoidal signal
k_period = 20  # number of samples for one period
sigma = 0.05  # standard deviation of weight Gaussian noise

y_dirac = gen_rand_pulse(K, n_pulses=N)  # random unit impulse generation
y_sin = gen_sinusoidal(K_sin, k_period)  # sinusoidal signal generation
y_noise = gen_wgn(K, sigma)  # weight Gaussian noise generation
y_baseline = gen_baseline_sin(K, k_period=int(K / 2))  # stochastic baseline generation

y = gen_convolve(y_dirac, y_sin) + y_noise + 2 * y_baseline  # design new signal

# -------- Compact (shorter) version ----------
K = 500
y = gen_convolve(gen_rand_pulse(K, n_pulses=N), gen_sinusoidal(K=20, k_period=20)) \
    + gen_wgn(K, sigma=0.05) \
    + 2 * gen_baseline_sin(K, k_period=int(K / 2))

# -------- Plotting ----------
fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y')
ax.plot(range(K), y)
plt.tight_layout()
plt.show()

(Source code)

Deterministic Signal Generators

gen_sinusoidal

gen_rectangle

Rectangular (pulse wave) signal generator

gen_triangle

Triangular signal generator

gen_slope

Slope signal generator

gen_pulse

Pulse signal generator

gen_baseline_sin

Baseline signal generator

gen_exponential

Exponentially decaying signal generator

gen_steps

Step signal generator

gen_slopes

Slopes signal generator

Random Signal Generators

gen_wgn

White Gaussian noise signal generator

gen_rand_slopes

Random ramps signal generator

gen_rand_pulse

Random pulse signal generator

gen_rand_walk

Random walk generator

Signal Modifiers

gen_convolve

Convolves two signals.

Helper Functions and Miscellaneous

k_period_to_omega

Converts sample base period (samples per cycle) to the normalized frequency

gen_multichannel

Returns a multi-channel signal out of arrays.

Test Signal Catalog (Collection of Recorded Bio-Signals)

The test signal catalog is a collection of recorded biological (e.g., electrocardiograms) and other signals, intended for development and educative purposes. The collection is extendable and each record is saved as a single .csv file in the directory lmlib/utils/data/.

Signal Catalog

The signal catalog is found here: Signal Catalog.

The files are directly available via Signal Catalog Access Functions.

Signal Catalog Access Functions

load_single_channel

Loads a single channel signal from the signal catalog, see Signal Catalog.

load_multi_channel

Loading a multi channel signal

Helper Functions and Miscellaneous (Beta State)

find_max_mask

Advanced peak finder with masked samples and search intervals.

load_source_csv

Loads time series from csv file with/without header section.

zero_cross_ind

Returns a list of the indeces of zero crossings

diff0

Gets difference of consecutive elements in vector, starting with the first element

edge_detection

Performs edge detection applying joint line models

poly_filter

Performs edge detection applying joint line models


API (moudle: utils)

gen_rectangle(K, k_period, k_on)

Rectangular (pulse wave) signal generator

Parameters
  • K (int) – Signal length

  • k_period (int) – periodicity, number of samples per period

  • k_on (int) – Number of samples of value 1, followed by k_period-k_on samples of value 0.

Returns

out (ndarray, shape=(K,)) – Returns a rectangular wave signal of length K.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rectangle(K, k_period=30, k_on=20)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Rectangle Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_rectangle_plot.png
gen_triangle(K, k_period)

Triangular signal generator

Parameters
  • K (int) – Signal length

  • k_period (int) – periodicity, number of samples per period

Returns

out (ndarray, shape=(K,)) – Returns a triangular signal of length K with k_period samples per triangle. Amplitudes aer normalize from 0 to 1.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_triangle(K, k_period=30)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Triangle Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_triangle_plot.png
gen_slope(K, k_period)

Slope signal generator

Parameters
  • K (int) – Signal length

  • k_period (int) – periodicity, number of samples per period

Returns

out (ndarray, shape=(K,)) – Returns a repetitive slope signal of length K. Amplitudes are normalize from 0 to 1.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_slope(K, k_period=23)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Slope Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_slope_plot.png
gen_pulse(K, ks)

Pulse signal generator

Parameters
  • K (int) – Signal length

  • ks (list) – Time indices of unit impulses

Returns

out (ndarray, shape=(K,)) – Returns a unit impulse signal trail of length K with values at indices ks set to 1, all others to 0.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 65
y = gen_pulse(K, ks=[10, 17, 59, 33])

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Unit Impulse Signal Generation')
ax.stem(range(K), y, use_line_collection=True)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_pulse_plot.png
gen_baseline_sin(K, k_period)

Baseline signal generator

The baseline is formed by the product of 4 sinusoidal. The highest frequency is given py k_period samples per cycle. The lower 3 frequencies are linear spaced from k_period to K. Each sinusoidal is weighted with its samples/period divided by K.

Parameters
  • K (int) – Signal length

  • k_period (int) – Normalized period of the highest frequency in the baseline in samples per cycle

Returns

out (ndarray, shape=(K,)) – Returns a baseline signal shaped by 4 sinusoidal of length K

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_baseline_sin(K, k_period=50)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Sinusoidal Baseline Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_baseline_sin_plot.png
gen_exponential(K, decay, k0=0)

Exponentially decaying signal generator

\(y_k = \gamma^{k-k_0}\)

Parameters
  • K (int) – Signal length

  • decay (float) – decay factor \(\gamma\)

  • k0 (int) – index shift \(k_0\); it follows that \(y_{k_0} = 1\)

Returns

out (numpy.ndarray) – Returns an exponential decaying signal of length K, normalized to 1 at index k0.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 70
decay = 0.95
k = 20
y = gen_exponential(K, decay, k)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Exponential Signal Generation')
ax.plot(range(K), y)
ax.scatter(k, y[k])

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_exponential_plot.png
gen_steps(K, ks, deltas)

Step signal generator

Parameters
  • K (int) – Signal length

  • ks (list) – Amplitude step locations (indeces)

  • deltas (list) – Relative step amplitudes at indeces ks

Returns

out (ndarray, shape=(K,)) – Returns a step signal of length K with steps of relative amplitudes deltas at indeces ks.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
ks = [5, 33, 50, 60, 77]
deltas = [-0.3, 0.6, 0.2, -0.5, -0.5]
y = gen_steps(K, ks, deltas)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Steps Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_steps_plot.png
gen_slopes(K, ks, deltas)

Slopes signal generator

Parameters
  • K (int) – Signal length

  • ks (list) – Indices of slope change

  • deltas (list) – Slope start to end difference at each index in ks

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with chances in slope by the values deltas at indeces ks.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
ks = [15, 33, 50, 60, 77]
deltas = [5, -2.5, -1, -3, 2]
y = gen_slopes(K, ks, deltas)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Slopes Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_slopes_plot.png
gen_wgn(K, sigma, seed=None)

White Gaussian noise signal generator

Parameters
  • K (int) – Signal length

  • sigma (float) – Sample variance

  • seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a white Gaussian noise signal of length K and variance sigma.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_wgn(K, sigma=0.5)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='White Gaussian Noise Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_wgn_plot.png
gen_rand_slopes(K, n_slopes, min_dist=0, seed=None)

Random ramps signal generator

Note: If seed is set to None, gen_rand_ramps() provides a random signal where the minimal distance between two edges is np.floor(0.05 * K) samples.

Parameters
  • K (int) – Signal length

  • n_slopes (int) – Number of slope changes in the signal

  • seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with N random changes in slope.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rand_slopes(K, n_slopes=4)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Slopes Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_rand_slopes_plot.png
gen_rand_pulse(K, n_pulses, length=1, seed=None)

Random pulse signal generator

Parameters
  • K (int) – Signal length

  • n_pulses (int) – Number of pulses in the signal

  • length (int) – pulse length (number of samples per pulse set to 1)

  • seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns signal of length K with exactly N unity pulses of length N at random positions.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 150
y = gen_rand_pulse(K, n_pulses=5, length=10)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Pulse Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_rand_pulse_plot.png
gen_rand_walk(K, seed=None)

Random walk generator

Parameters
  • K (int) – Signal length

  • seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with a random walk

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rand_walk(K)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Walk Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/gen_rand_walk_plot.png
gen_convolve(base, template)

Convolves two signals. The output signal shape (number of channels and signal length) is preserved from the base signal.

Parameters
  • base (array_like) – Base signal to be convolved, either single- or multi-channel.

  • template (array_like) – Signal template to be convolved with base, either a single- or multi-channel. If base is multi-channel, the number of channels has to correspond to the number of channels of base.

Returns

out (ndarray, shape=(K,)) – If template is a sigle-channel signal, the convolution is applied to each channel of base, otherwise the convolution between base and template is applied per-channel. The output signal is of the same dimension as base signal, cf. numpy.convolve(..., mode='same').

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 200
y_impulse = gen_rand_pulse(K, n_pulses=4)
y_template = gen_sinusoidal(K=10, k_period=10)
y = gen_convolve(y_impulse, y_template)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Convolve Random Unit Impulse Signal with Sinusoidal')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code)

load_single_channel(name, K, k=0, ch_select=0)

Loads a single channel signal from the signal catalog, see Signal Catalog.

Parameters
  • name (str) – Signal name (from signal catalog)

  • K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

  • k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

  • ch_select (int) – Selects channel index to be loaded from multi-channel signals. 0 <= ch_select < M, where M is the number of channel in the filename.

Returns

out (ndarray, shape=(K,)) – Single-channel signal with shape=(K,)

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 6000
file_name = 'EECG_BASELINE_1CH_10S_FS2400HZ.csv'
y = load_single_channel(file_name, K)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title=file_name)
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/load_single_channel_plot_00_00.png
# Load a single channel signal out of a multi-channel file
K = 6000
file_name = 'EECG_BASELINE_9CH_10S_FS2400HZ.csv'
y = load_single_channel(file_name, K, ch_select=4)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title=file_name)
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

(png, hires.png, pdf)

../_images/load_single_channel_plot_01_00.png
load_multi_channel(name, K, k=0, ch_select=None)

Loading a multi channel signal

Parameters
  • name (str) – Signal name (from signal catalog)

  • K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

  • k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

  • ch_select (list of length M, None) – List of channels to be loaded. Selects channel indices to be loaded from multi-channel signals. 0 <= ch_select < M, where M is the number of channel in the filename. If set to None, all channels are loaded.

Returns

out (ndarray, shape=(K, M)) – Multi-channel signal of shape=(K, M), where M is the number of channels

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 6000
file_name = 'EECG_BASELINE_9CH_10S_FS2400HZ.csv'
cs = [0, 1, 2]
y = load_multi_channel(file_name, K, ch_select=cs)

fig, axs = plt.subplots(len(cs), 1, figsize=(6, 6))
for n, ch in enumerate(cs):
    axs[n].set(xlabel='k', ylabel=f'channel {ch}')
    axs[n].plot(range(K), y[:, n])
axs[0].set_title(file_name)
plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

../_images/load_multi_channel_plot.png
k_period_to_omega(k_period)

Converts sample base period (samples per cycle) to the normalized frequency

Parameters

k_period (int) – number of samples per period

Returns

w (float) – Normalized frequency, \(\omega = {2 \pi}/{k_\mathrm{period}}\)

gen_multichannel(arrays, option='multi-output')

Returns a multi-channel signal out of arrays. Either the signal is multi-output of shape (K, L) where L is the number of channels. Or the signal has multi-set (parallel) which has a shape of (K, 1, S), where S is the number if channels.

Parameters
  • arrays (array_like) – Set of single-channel arrays

  • option (str) – ‘multi-output’` for a signal shape of (K, L) or 'multi-set' for a signal shape of (K, 1, S)

Returns

signal (ndarray) – Multi-channel signal

find_max_mask(y, msks, locs=None, range_start=None, range_end=None, threshold=-inf, direction='maximum', skip_invalid=False, SHOW_DEBUG_PLOT=False)

Advanced peak finder with masked samples and search intervals.

This peak finder finds in signal y for every interval that sample fulfilling multiple criteria.

Parameters
  • y (array_like, shape=(K,)) – input signal

  • msks (array_like of shape=(K) of Boolean OR tuple of array_like of shape=(K) of Boolean,) – a mask or a list of masks each of length K with True for any valid samples. If multiple masks are provided, a sample is only valid, if set True in all masks (multiple mask are merged using logic AND). The peak index is decided among all valid samples

  • locs (array_line of shape=(P) of int,) – reference index for P intervals

  • range_start (int) – start index of a interval relative to the reference index

  • range_end (int) – end index of a interval relative to the reference index

  • threshold (int) – min. value in y for peaks

  • direction (string {"forward", "backward", "maximum"}) – search direction "forward" or "backward" (return first value fulfilling all criteria) or "maximum" (take maximum value of y)

  • skip_invalid (bool) –

    True: removes entries if no peak is found in a certain interval. False: if no peak is found in a certain interval, the index is set to -1 instead.

    default: False

Returns

out (ndarray of shape(<=K,) of int) – index per location interval or -1 if skip_invalid == False. if skip_invalid == True, the returned vector might has less than K entries.

load_source_csv(filename, time_format=None, K=-1, k=0, ch_select=None)

Loads time series from csv file with/without header section.

File format:

# Description Line 1
# ...
# Description Line n
#
time,         signal1, signal2, ...
00:00:20.000, 10.34,   11.09, ...
  • Description: the file can start with an (optional) description section, prefixed with an ‘#’

  • Header: The data table can start with an (optional) column labels. A header line is detected, if the first data line starts with a non-nummeric character

  • Data: float values separated by either ',',';','\space','\tab'; first column commonly contains the time; either as float (seconds) or as ‘hh:mm:ss.ttt’; the later will be converted to floats (in seconds).

Parameters
  • filename (string) – csv file to be loaded

  • time_format (string) – Format string to be used to decode first line: “H:M:S” (e.g., 00:02:01.123) or None (e.g. 1.12)

  • K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

  • k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

  • ch_select (int, list, None) – int: channel index to be loaded in single channel mode. list: List of M channels to be loaded in multi channel mode. Selects channel indices to be loaded from multi-channel signals. 0 <= ch_select < Number of CSV columns. If set to None, all channels are loaded in multi channel mode and M = Number of CSV columns. First column in the CSV file is addressed as channel 0, and contains the time, if time_format is set accordingly.

Returns

out (Tuple (col_labels, data)) –

with
  • col_labels: list of column labels; if csv does not provide column headers, headers are set as 'col1','col2','col3', ... * NOT YET IMPLEMENTED *

  • data: (ndarray, shape=(K[, M])) – Multi-channel signal of shape=(K, M) if ch_select is of type list, single-channel signal of shape=(K, ) if ch_select is of type `int.

zero_cross_ind(y, threshold=0)

Returns a list of the indeces of zero crossings

Parameters
  • y (array_like of shape=(K,) of floats) – signal vector to detect zero-crossings

  • threshold (float) – crossing threshold (default: 0).

Returns

out (ndarray of ints) – array with indices i where y[i]<0 and y[i+1]>=0 (i.e., indices before the threshold).

diff0(y)

Gets difference of consecutive elements in vector, starting with the first element

This is the same as numpy.diff(y), but starting with the first element y[0] as the first difference (leading to a vector of the same length as the input signal vector y.

Implementation:

return np.append(np.array(y[0]), np.diff(y))
Parameters

y (array_like of shape(K,) of floats) – input signal vector

Returns

out (ndarray of shape(K,) of floats) – element-wise differences

edge_detection(y, ab, g)

Performs edge detection applying joint line models

Uses a two-line model to detect edges, according to example: Edge Detection [ex110.0]

Parameters
  • y (array_like of shape=(K,) of floats) – observation vector

  • ab (tuple (a,b)) –

    • a (int, >0): left-sided window length (>0)

    • b (int, >0): right-sided window length (>0)

  • g (tuple (g_a, g_b) of float) –

    Effective number of samples under the left-sided and right-sided window, see CostSegment.
    • g_a (float, >0): left-sided window length (>0)

    • g_b (float, >0): right-sided window length (>0)

  • g_a (float) – Effective number of samples under the right-sided window, see CostSegment.

Returns

  • lcr (ndarray of shape(K,) of floats) – log-likelihood ratio for a detected edge

  • y0 (ndarray of shape(K,) of floats) – position estimate of edge on y-axis

  • a0 (ndarray of shape(K,) of floats) – line slope of left-sided model

  • a1 (ndarray of shape(K,) of floats) – line slope of right-sided model

poly_filter(y, ab, g, poly_degree=0)

Performs edge detection applying joint line models

Uses polynomial of oder ‘poly_degree’ to filter signal. (for poly_degree=0, the filter leads to a moving average filter) example: sphx_glr_generated_examples_12-lssm-costs-filters-example-ex121.0-sc-filter.py

Parameters
  • y (array_like of shape=(K,) of floats) – observation vector

  • ab (tuple (a,b)) –

    • a (int, >0): left-sided window length (>0)

    • b (int, >0): right-sided window length (>0)

  • g (tuple (g_a, g_b) of float) –

    Effective number of samples under the left-sided and right-sided window, see CostSegment.
    • g_a (float, >0): left-sided window length (>0)

    • g_b (float, >0): right-sided window length (>0)

  • g_a (float) – Effective number of samples under the right-sided window, see CostSegment.

  • poly_degree (int) – degree of polynomial to fit; default: 0 (constant value, i.e., moving average)

Returns

  • y0 (ndarray of shape(K,) of floats) – average estimate

  • error (ndarray of shape(K,) of floats) – squared error of fit