Synthetic Data Simulator

33 min

Overview

What is this tool?

The Synthetic Data Simulator is a web-based application that helps you generate realistic time-series data for testing, research, or development purposes. You can create clean mathematical signals and add various types of noise to simulate real-world data conditions.

Getting Started

Accessing the Application

Run on any LitmusEdge device: docker run -d -p 8501:8501 litmusedge.azurecr.io/litmus/synthetic-sims:latest

Navigate to the {edgeUrl}:8501

The interface will load automatically

How to Generate Signals

Step 1: Choose Your Base Signal

Select from 20 different signal types:

Stochastic Signals:

Random Walk: Models cumulative random processes (financial data, sensor drift)

White Noise Signal: Uncorrelated random values with normal distribution

Periodic Signals:

Sine Cosine Equation: Combined sinusoidal functions for complex periodic behavior

Square Wave: Binary state transitions with defined duty cycles

Triangle Wave: Linear rise-and-fall patterns

Sawtooth Wave: Asymmetric periodic ramps

Communication & Signal Processing:

Chirp Signal: Frequency-swept signals for radar and communications testing

Amplitude Modulated Sine Wave: Carrier wave with amplitude modulation

Square Pulse Train: Periodic pulses with adjustable duty cycle

Mathematical Functions:

Gaussian Function: Bell curve distribution

Lorentzian Function: Resonance curve modeling

Sinc Function: Cardinal sine function for digital signal processing

Step Function: Discontinuous transitions

Impulse Function: Single-point response testing

System Response Patterns:

Damped Oscillation: Exponentially decaying oscillations

Exponential Decay: Pure exponential decline

Hyperbolic Tangent: S-curve transitions

Logarithmic Signal: Logarithmic growth patterns

Unit Ramp: Linear increase over time

Rectified Sine Wave: Full-wave rectified sinusoid

Step 2: Set Time Parameters

Start Time: When your signal begins (in milliseconds)

End Time: When your signal ends (in milliseconds)

Number of Points: How many data points to generate

Time Interval Calculation: The interval between points equals (End Time - Start Time) ÷ (Number of Points - 1). For 1-second intervals, set Number of Points to (End Time - Start Time) + 1.

Step 3: Add Realistic Noise (Optional)

Choose from 12 different noise types:

Statistical Noise Models:

Gaussian: Normal distribution additive noise (most common)

Pink Noise: 1/f frequency characteristic noise (natural-looking variations)

Uniform Noise: Evenly distributed random variations

Random Walk Noise: Cumulative drift simulation

Physical Interference Models:

Burst Noise: Intermittent high-amplitude interference

Harmonic Noise: Sinusoidal interference at specified frequencies

Exponentially Decaying Noise: Time-variant noise with exponential decay

Data Quality Issues:

Random Outliers: Probabilistic anomalous readings

Horrible Outliers: Extreme outlier simulation for stress testing

Gaps in Data: Missing data point simulation

Systematic Patterns:

Seasonality: Periodic variations based on signal duration

Damped Sine Wave Noise: Decaying oscillatory interference

Step 4: Generate Your Signal

Click "Generate Signal"

View the interactive plot showing both original and noisy signals

Download your data as CSV files

Advanced Features

Creating Multiple Signals

Click "Add New Entry" to create additional signal generators

Each entry operates independently

Configure different signals with different noise combinations

Real-Time Data Streaming via NATS

Enter a Topic Name for each signal you want to stream

Configure your signal and noise parameters

Click "Start Publishing to NATS"

Data publishes at 1-second intervals to the configured NATS server

Message Format:

{
  "timestamp": 1234567890,
  "value": 42.5
}

Noise Application Process

Noise functions are applied sequentially. Each noise type modifies the result from the previous application, creating cumulative effects. The system automatically calculates noise strength as 10% of the original signal's amplitude range.

Use Cases

Algorithm Development & Testing

Baseline Testing: Start with clean signals to verify basic functionality

Robustness Testing: Gradually add noise types to test algorithm resilience

Stress Testing: Use "Horrible Outliers" for extreme condition simulation

Missing Data Handling: Apply "Gaps in Data" to test interpolation algorithms

IoT & Sensor Simulation

Realistic Sensor Data: Combine Random Walk with Gaussian noise

Network Issues: Add "Gaps in Data" for connectivity problems

Interference Modeling: Use Burst or Harmonic noise for electromagnetic interference

Drift Simulation: Apply Random Walk noise for sensor calibration drift

Financial & Time Series Analysis

Price Movement Simulation: Random Walk with Burst noise for volatility

Market Closure Simulation: Gaps in Data for non-trading periods

Seasonal Effects: Apply Seasonality noise for cyclical patterns

Algorithm Backtesting: Generate consistent test datasets

System Validation & Demonstrations

Consistent Demonstrations: Reproducible datasets for stakeholder presentations

Performance Benchmarking: Compare algorithm performance across known datasets

Training Data Generation: Labeled datasets with known ground truth

Troubleshooting

Common Issues

"Start time must be less than End time": Verify End Time > Start Time

Plot not displaying: Click "Generate Signal" after parameter changes

No download available: Generate signal before attempting CSV export

NATS publishing fails: Ensure Topic Name is entered and signal is generated

Getting Support

Use "Reset Inputs" to clear session and restart

Contact system administrator for NATS connectivity issues

Verify Docker container is running with proper port mapping

Data Export Options

Original Signal CSV: Clean mathematical function output

Noisy Signal CSV: Signal after noise application

Real-time NATS Stream: Live data publishing for integration testing

Generated data is compatible with standard analysis tools, databases, and any system accepting CSV format or NATS messaging.

Mathematical Function Definitions

Original Signal Functions

Random Walk

f(n) = cumsum(ε_i) where ε_i ~ N(0, 1)

Sine Cosine Equation

f(n) = 5 sin(2πn) + 3.6 cos(25πn)

Square Wave

f(n) = square(2π · 5 · n)

Sawtooth Wave

f(n) = sawtooth(2π · 5 · n)

Chirp Signal

f(n) = chirp(n, f₀=1, f₁=10, t₁=n_max, method="linear")

Damped Oscillation

f(n) = e^(-n/10) sin(2πn)

Exponential Decay

f(n) = e^(-n)

Triangle Wave

f(n) = sawtooth(2π · 5 · n, width=0.5)

Step Function

f(n) = {1 if n ≥ (n₀ + n_max)/2, 0 otherwise}

Impulse Function

f(n) = {1 if i = len(n)//2, 0 otherwise}

White Noise Signal

f(n) = ε_i where ε_i ~ N(0, 1)

Sinc Function

f(n) = sinc(n - (n₀ + n_max)/2)

Rectified Sine Wave

f(n) = |sin(2πn)|

Amplitude Modulated Sine Wave

f_c = 5, f_m = 0.5, m = 0.7
f(n) = (1 + m sin(2πf_m n)) sin(2πf_c n)

Square Pulse Train

f(n) = square(2πn, duty=0.1)

Lorentzian Function

γ = 1
f(n) = γ / (π(n² + γ²))

Gaussian Function

σ = 1
f(n) = (1/(σ√(2π))) e^(-(n²)/(2σ²))

Logarithmic Signal

f(n) = ln(n + 1)

Hyperbolic Tangent

f(n) = tanh(n)

Unit Ramp

f(n) = n

Noise Functions

Gaussian Noise

f_noisy(n) = f_orig(n) + α · ε where ε ~ N(0, 1)

Pink Noise

ε ~ N(0, 1)
[b, a] = butter(1, 0.1)
pink = lfilter(b, a, ε)
f_noisy(n) = f_orig(n) + α · pink

Exponentially Decaying Noise

β = 0.5 (default decay rate)
f_noisy(n) = f_orig(n) + α · e^(-βn) · ε where ε ~ N(0, 1)

Burst Noise

p = 0.01 (default burst probability)
A = 5 (default burst amplitude)
burst ~ Bernoulli(p) × A
f_noisy(n) = f_orig(n) + α · burst

Random Walk Noise

f_noisy(n) = f_orig(n) + α · cumsum(ε_i) where ε_i ~ N(0, 1)

Uniform Noise

low = -1, high = 1 (default bounds)
ε ~ U(low, high)
f_noisy(n) = f_orig(n) + α · ε

Harmonic Noise

f_h = 2.0 (default harmonic frequency)
f_noisy(n) = f_orig(n) + α · sin(2πf_h n)

Random Outliers

p = 0.01 (default outlier probability)
A = 5 (default outlier amplitude)
outliers ~ Bernoulli(p) × A
f_noisy(n) = f_orig(n) + α · outliers

Horrible Outliers

p = 0.01 (default outlier probability)
A = 50 (default large outlier amplitude)
outliers ~ Bernoulli(p) × A
f_noisy(n) = f_orig(n) + α · outliers

Seasonality Noise

T = n_max - n_min (period)
A = 2 (default amplitude)
f_noisy(n) = f_orig(n) + α · A · sin(2πn/T)

Damped Sine Wave Noise

δ = 0.05 (default damping factor)
f = 1.0 (default frequency)
f_noisy(n) = f_orig(n) + e^(-δn) sin(2πfn)

Gaps in Data

p = 0.05 (default gap probability)
gaps ~ Bernoulli(p)
f_noisy(n) = {NaN if gaps = 1, f_orig(n) otherwise}

Noise Strength Calculation

α = 0.1 × (max(f_orig) - min(f_orig))

Where α is the noise strength parameter applied to all noise functions that accept it.