# module: utils¶

Collection of practical utils, such as deterministic and stochastic signals generators and a collection of recorded biological signals, see Signal Catalog. This utils are intended to accelerate algorithm development and/or for educative purposes.

## Signal Generators¶

Class of (test) signal generators for often used signals.

### Introductory Example¶

Example demonstrates use of signal generators in a explanatory (longer) or a compact (shorter) version.

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

# -------- Explanatory (longer) version ----------
K = 500  # number of sample of the signal
N = 5  # number of unit impulses
K_sin = 20  # number of sample of the sinusoidal signal
k_period = 20  # number of samples for one period
sigma = 0.05  # standard deviation of weight Gaussian noise

y_dirac = gen_rand_pulse(K, n_pulses=N)  # random unit impulse generation
y_sin = gen_sinusoidal(K_sin, k_period)  # sinusoidal signal generation
y_noise = gen_wgn(K, sigma)  # weight Gaussian noise generation
y_baseline = gen_baseline_sin(K, k_period=int(K / 2))  # stochastic baseline generation

y = gen_convolve(y_dirac, y_sin) + y_noise + 2 * y_baseline  # design new signal

# -------- Compact (shorter) version ----------
K = 500
y = gen_convolve(gen_rand_pulse(K, n_pulses=N), gen_sinusoidal(K=20, k_period=20)) \
+ gen_wgn(K, sigma=0.05) \
+ 2 * gen_baseline_sin(K, k_period=int(K / 2))

# -------- Plotting ----------
fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y')
ax.plot(range(K), y)
plt.tight_layout()
plt.show()


### Deterministic Signal Generators¶

 gen_sinusoidal gen_rectangle Rectangular (pulse wave) signal generator gen_triangle Triangular signal generator gen_slope Slope signal generator gen_pulse Pulse signal generator gen_baseline_sin Baseline signal generator gen_exponential Exponentially decaying signal generator gen_steps Step signal generator gen_slopes Slopes signal generator

### Random Signal Generators¶

 gen_wgn White Gaussian noise signal generator gen_rand_slopes Random ramps signal generator gen_rand_pulse Random pulse signal generator gen_rand_walk Random walk generator

### Signal Modifiers¶

 gen_convolve Convolves two signals.

### Helper Functions and Miscellaneous¶

 k_period_to_omega Converts sample base period (samples per cycle) to the normalized frequency gen_multichannel Returns a multi-channel signal out of arrays.

## Test Signal Catalog (Collection of Recorded Bio-Signals)¶

The test signal catalog is a collection of recorded biological (e.g., electrocardiograms) and other signals, intended for development and educative purposes. The collection is extendable and each record is saved as a single .csv file in the directory lmlib/utils/data/.

### Signal Catalog¶

The signal catalog is found here: Signal Catalog.

The files are directly available via Signal Catalog Access Functions.

### Signal Catalog Access Functions¶

 load_single_channel Loads a single channel signal from the signal catalog, see Signal Catalog. load_multi_channel Loading a multi channel signal

## Helper Functions and Miscellaneous (Beta State)¶

 find_max_mask Advanced peak finder with masked samples and search intervals. load_source_csv Loads time series from csv file with/without header section. zero_cross_ind Returns a list of the indeces of zero crossings diff0 Gets difference of consecutive elements in vector, starting with the first element edge_detection Performs edge detection applying joint line models poly_filter Performs edge detection applying joint line models

## API (moudle: utils)¶

gen_rectangle(K, k_period, k_on)

Rectangular (pulse wave) signal generator

Parameters
• K (int) – Signal length

• k_period (int) – periodicity, number of samples per period

• k_on (int) – Number of samples of value 1, followed by k_period-k_on samples of value 0.

Returns

out (ndarray, shape=(K,)) – Returns a rectangular wave signal of length K.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rectangle(K, k_period=30, k_on=20)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Rectangle Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_triangle(K, k_period)

Triangular signal generator

Parameters
• K (int) – Signal length

• k_period (int) – periodicity, number of samples per period

Returns

out (ndarray, shape=(K,)) – Returns a triangular signal of length K with k_period samples per triangle. Amplitudes aer normalize from 0 to 1.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_triangle(K, k_period=30)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Triangle Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_slope(K, k_period)

Slope signal generator

Parameters
• K (int) – Signal length

• k_period (int) – periodicity, number of samples per period

Returns

out (ndarray, shape=(K,)) – Returns a repetitive slope signal of length K. Amplitudes are normalize from 0 to 1.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_slope(K, k_period=23)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Slope Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_pulse(K, ks)

Pulse signal generator

Parameters
• K (int) – Signal length

• ks (list) – Time indices of unit impulses

Returns

out (ndarray, shape=(K,)) – Returns a unit impulse signal trail of length K with values at indices ks set to 1, all others to 0.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 65
y = gen_pulse(K, ks=[10, 17, 59, 33])

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Unit Impulse Signal Generation')
ax.stem(range(K), y, use_line_collection=True)

plt.tight_layout()
plt.show() gen_baseline_sin(K, k_period)

Baseline signal generator

The baseline is formed by the product of 4 sinusoidal. The highest frequency is given py k_period samples per cycle. The lower 3 frequencies are linear spaced from k_period to K. Each sinusoidal is weighted with its samples/period divided by K.

Parameters
• K (int) – Signal length

• k_period (int) – Normalized period of the highest frequency in the baseline in samples per cycle

Returns

out (ndarray, shape=(K,)) – Returns a baseline signal shaped by 4 sinusoidal of length K

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_baseline_sin(K, k_period=50)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Sinusoidal Baseline Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_exponential(K, decay, k0=0)

Exponentially decaying signal generator

$$y_k = \gamma^{k-k_0}$$

Parameters
• K (int) – Signal length

• decay (float) – decay factor $$\gamma$$

• k0 (int) – index shift $$k_0$$; it follows that $$y_{k_0} = 1$$

Returns

out (numpy.ndarray) – Returns an exponential decaying signal of length K, normalized to 1 at index k0.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 70
decay = 0.95
k = 20
y = gen_exponential(K, decay, k)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Exponential Signal Generation')
ax.plot(range(K), y)
ax.scatter(k, y[k])

plt.tight_layout()
plt.show() gen_steps(K, ks, deltas)

Step signal generator

Parameters
• K (int) – Signal length

• ks (list) – Amplitude step locations (indeces)

• deltas (list) – Relative step amplitudes at indeces ks

Returns

out (ndarray, shape=(K,)) – Returns a step signal of length K with steps of relative amplitudes deltas at indeces ks.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
ks = [5, 33, 50, 60, 77]
deltas = [-0.3, 0.6, 0.2, -0.5, -0.5]
y = gen_steps(K, ks, deltas)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Steps Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_slopes(K, ks, deltas)

Slopes signal generator

Parameters
• K (int) – Signal length

• ks (list) – Indices of slope change

• deltas (list) – Slope start to end difference at each index in ks

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with chances in slope by the values deltas at indeces ks.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
ks = [15, 33, 50, 60, 77]
deltas = [5, -2.5, -1, -3, 2]
y = gen_slopes(K, ks, deltas)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Slopes Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_wgn(K, sigma, seed=None)

White Gaussian noise signal generator

Parameters
• K (int) – Signal length

• sigma (float) – Sample variance

• seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a white Gaussian noise signal of length K and variance sigma.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_wgn(K, sigma=0.5)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='White Gaussian Noise Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_rand_slopes(K, n_slopes, min_dist=0, seed=None)

Random ramps signal generator

Note: If seed is set to None, gen_rand_ramps() provides a random signal where the minimal distance between two edges is np.floor(0.05 * K) samples.

Parameters
• K (int) – Signal length

• n_slopes (int) – Number of slope changes in the signal

• seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with N random changes in slope.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rand_slopes(K, n_slopes=4)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Slopes Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_rand_pulse(K, n_pulses, length=1, seed=None)

Random pulse signal generator

Parameters
• K (int) – Signal length

• n_pulses (int) – Number of pulses in the signal

• length (int) – pulse length (number of samples per pulse set to 1)

• seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns signal of length K with exactly N unity pulses of length N at random positions.

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 150
y = gen_rand_pulse(K, n_pulses=5, length=10)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Pulse Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_rand_walk(K, seed=None)

Random walk generator

Parameters
• K (int) – Signal length

• seed (int, None) – random number generator seed, default = None.

Returns

out (ndarray, shape=(K,)) – Returns a signal of length K with a random walk

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 100
y = gen_rand_walk(K)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Random Walk Signal Generation')
ax.plot(range(K), y)

plt.tight_layout()
plt.show() gen_convolve(base, template)

Convolves two signals. The output signal shape (number of channels and signal length) is preserved from the base signal.

Parameters
• base (array_like) – Base signal to be convolved, either single- or multi-channel.

• template (array_like) – Signal template to be convolved with base, either a single- or multi-channel. If base is multi-channel, the number of channels has to correspond to the number of channels of base.

Returns

out (ndarray, shape=(K,)) – If template is a sigle-channel signal, the convolution is applied to each channel of base, otherwise the convolution between base and template is applied per-channel. The output signal is of the same dimension as base signal, cf. numpy.convolve(..., mode='same').

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 200
y_impulse = gen_rand_pulse(K, n_pulses=4)
y_template = gen_sinusoidal(K=10, k_period=10)
y = gen_convolve(y_impulse, y_template)

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title='Convolve Random Unit Impulse Signal with Sinusoidal')
ax.plot(range(K), y)

plt.tight_layout()
plt.show()

load_single_channel(name, K, k=0, ch_select=0)

Loads a single channel signal from the signal catalog, see Signal Catalog.

Parameters
• name (str) – Signal name (from signal catalog)

• K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

• k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

• ch_select (int) – Selects channel index to be loaded from multi-channel signals. 0 <= ch_select < M, where M is the number of channel in the filename.

Returns

out (ndarray, shape=(K,)) – Single-channel signal with shape=(K,)

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 6000
file_name = 'EECG_BASELINE_1CH_10S_FS2400HZ.csv'

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title=file_name)
ax.plot(range(K), y)

plt.tight_layout()
plt.show() # Load a single channel signal out of a multi-channel file
K = 6000
file_name = 'EECG_BASELINE_9CH_10S_FS2400HZ.csv'

fig, ax = plt.subplots(figsize=(6, 3))
ax.set(xlabel='k', ylabel='y', title=file_name)
ax.plot(range(K), y)

plt.tight_layout()
plt.show()


(png, hires.png, pdf) load_multi_channel(name, K, k=0, ch_select=None)

Parameters
• name (str) – Signal name (from signal catalog)

• K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

• k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

• ch_select (list of length M, None) – List of channels to be loaded. Selects channel indices to be loaded from multi-channel signals. 0 <= ch_select < M, where M is the number of channel in the filename. If set to None, all channels are loaded.

Returns

out (ndarray, shape=(K, M)) – Multi-channel signal of shape=(K, M), where M is the number of channels

Example

import matplotlib.pyplot as plt
from lmlib.utils.generator import *

K = 6000
file_name = 'EECG_BASELINE_9CH_10S_FS2400HZ.csv'
cs = [0, 1, 2]

fig, axs = plt.subplots(len(cs), 1, figsize=(6, 6))
for n, ch in enumerate(cs):
axs[n].set(xlabel='k', ylabel=f'channel {ch}')
axs[n].plot(range(K), y[:, n])
axs.set_title(file_name)
plt.tight_layout()
plt.show() k_period_to_omega(k_period)

Converts sample base period (samples per cycle) to the normalized frequency

Parameters

k_period (int) – number of samples per period

Returns

w (float) – Normalized frequency, $$\omega = {2 \pi}/{k_\mathrm{period}}$$

gen_multichannel(arrays, option='multi-output')

Returns a multi-channel signal out of arrays. Either the signal is multi-output of shape (K, L) where L is the number of channels. Or the signal has multi-set (parallel) which has a shape of (K, 1, S), where S is the number if channels.

Parameters
• arrays (array_like) – Set of single-channel arrays

• option (str) – ‘multi-output’ for a signal shape of (K, L) or 'multi-set' for a signal shape of (K, 1, S)

Returns

signal (ndarray) – Multi-channel signal

find_max_mask(y, msks, locs=None, range_start=None, range_end=None, threshold=-inf, direction='maximum', skip_invalid=False, SHOW_DEBUG_PLOT=False)

This peak finder finds in signal y for every interval that sample fulfilling multiple criteria.

Parameters
• y (array_like, shape=(K,)) – input signal

• msks (array_like of shape=(K) of Boolean OR tuple of array_like of shape=(K) of Boolean,) – a mask or a list of masks each of length K with True for any valid samples. If multiple masks are provided, a sample is only valid, if set True in all masks (multiple mask are merged using logic AND). The peak index is decided among all valid samples

• locs (array_line of shape=(P) of int,) – reference index for P intervals

• range_start (int) – start index of a interval relative to the reference index

• range_end (int) – end index of a interval relative to the reference index

• threshold (int) – min. value in y for peaks

• direction (string {"forward", "backward", "maximum"}) – search direction "forward" or "backward" (return first value fulfilling all criteria) or "maximum" (take maximum value of y)

• skip_invalid (bool) –

True: removes entries if no peak is found in a certain interval. False: if no peak is found in a certain interval, the index is set to -1 instead.

default: False

Returns

out (ndarray of shape(<=K,) of int) – index per location interval or -1 if skip_invalid == False. if skip_invalid == True, the returned vector might has less than K entries.

load_source_csv(filename, time_format=None, K=-1, k=0, ch_select=None)

File format:

# Description Line 1
# ...
# Description Line n
#
time,         signal1, signal2, ...
00:00:20.000, 10.34,   11.09, ...

• Description: the file can start with an (optional) description section, prefixed with an ‘#’

• Header: The data table can start with an (optional) column labels. A header line is detected, if the first data line starts with a non-nummeric character

• Data: float values separated by either ',',';','\space','\tab'; first column commonly contains the time; either as float (seconds) or as ‘hh:mm:ss.ttt’; the later will be converted to floats (in seconds).

Parameters
• filename (string) – csv file to be loaded

• time_format (string) – Format string to be used to decode first line: “H:M:S” (e.g., 00:02:01.123) or None (e.g. 1.12)

• K (int) – Length of signal to be loaded or -1 to load until end of file. If K is larger than the maximal signal length, an assertion is raised.

• k (int) – Signal load start index. If k is larger than the maximal signal length, an assertion is raised.

• ch_select (int, list, None) – int: channel index to be loaded in single channel mode. list: List of M channels to be loaded in multi channel mode. Selects channel indices to be loaded from multi-channel signals. 0 <= ch_select < Number of CSV columns. If set to None, all channels are loaded in multi channel mode and M = Number of CSV columns. First column in the CSV file is addressed as channel 0, and contains the time, if time_format is set accordingly.

Returns

out (Tuple (col_labels, data)) –

with
• col_labels: list of column labels; if csv does not provide column headers, headers are set as 'col1','col2','col3', ... * NOT YET IMPLEMENTED *

• data: (ndarray, shape=(K[, M])) – Multi-channel signal of shape=(K, M) if ch_select is of type list, single-channel signal of shape=(K, ) if ch_select is of type int.

zero_cross_ind(y, threshold=0)

Returns a list of the indeces of zero crossings

Parameters
• y (array_like of shape=(K,) of floats) – signal vector to detect zero-crossings

• threshold (float) – crossing threshold (default: 0).

Returns

out (ndarray of ints) – array with indices i where y[i]<0 and y[i+1]>=0 (i.e., indices before the threshold).

diff0(y)

Gets difference of consecutive elements in vector, starting with the first element

This is the same as numpy.diff(y), but starting with the first element y as the first difference (leading to a vector of the same length as the input signal vector y.

Implementation:

return np.append(np.array(y), np.diff(y))

Parameters

y (array_like of shape(K,) of floats) – input signal vector

Returns

out (ndarray of shape(K,) of floats) – element-wise differences

edge_detection(y, ab, g)

Performs edge detection applying joint line models

Uses a two-line model to detect edges, according to example: Edge Detection [ex110.0]

Parameters
• y (array_like of shape=(K,) of floats) – observation vector

• ab (tuple (a,b)) –

• a (int, >0): left-sided window length (>0)

• b (int, >0): right-sided window length (>0)

• g (tuple (g_a, g_b) of float) –

Effective number of samples under the left-sided and right-sided window, see CostSegment.
• g_a (float, >0): left-sided window length (>0)

• g_b (float, >0): right-sided window length (>0)

• g_a (float) – Effective number of samples under the right-sided window, see CostSegment.

Returns

• lcr (ndarray of shape(K,) of floats) – log-likelihood ratio for a detected edge

• y0 (ndarray of shape(K,) of floats) – position estimate of edge on y-axis

• a0 (ndarray of shape(K,) of floats) – line slope of left-sided model

• a1 (ndarray of shape(K,) of floats) – line slope of right-sided model

poly_filter(y, ab, g, poly_degree=0)

Performs edge detection applying joint line models

Uses polynomial of oder ‘poly_degree’ to filter signal. (for poly_degree=0, the filter leads to a moving average filter) example: sphx_glr_generated_examples_12-lssm-costs-filters-example-ex121.0-sc-filter.py

Parameters
• y (array_like of shape=(K,) of floats) – observation vector

• ab (tuple (a,b)) –

• a (int, >0): left-sided window length (>0)

• b (int, >0): right-sided window length (>0)

• g (tuple (g_a, g_b) of float) –

Effective number of samples under the left-sided and right-sided window, see CostSegment.
• g_a (float, >0): left-sided window length (>0)

• g_b (float, >0): right-sided window length (>0)

• g_a (float) – Effective number of samples under the right-sided window, see CostSegment.

• poly_degree (int) – degree of polynomial to fit; default: 0 (constant value, i.e., moving average)

Returns

• y0 (ndarray of shape(K,) of floats) – average estimate

• error (ndarray of shape(K,) of floats) – squared error of fit