Sensor API Reference

The deeplens.sensor module simulates the image formation pipeline after the lens: photoelectric conversion, Bayer CFA, noise, and the full ISP chain (demosaicing, white balance, colour correction, gamma). All components are differentiable via PyTorch.

Sensor Types

Sensor	Description
Sensor	Minimal base class with noise model and basic gamma
RGBSensor	Bayer CFA + shot/read noise + fully invertible ISP
MonoSensor	Monochrome sensor without colour filter array
EventSensor	Event-based (DVS) sensor

Sensor (base)

deeplens.sensor.Sensor

Sensor(size=(8.0, 6.0), res=(4000, 3000))

Bases: Module

Minimal image sensor with gamma-only ISP.

The simplest sensor model: records physical size and resolution, and applies only a gamma correction in the ISP forward pass. For a sensor with noise simulation and Bayer demosaicing use :class:~deeplens.sensor.rgb_sensor.RGBSensor.

Attributes:

Name	Type	Description
`size`	`tuple`	Physical sensor size (W, H) [mm].
`res`	`tuple`	Pixel resolution (W, H).
`isp`	`Sequential`	ISP pipeline (`GammaCorrection` by default).

Example

sensor = Sensor(size=(8.0, 6.0), res=(4000, 3000)) sensor = Sensor.from_config("sensor.json")

Initialize a minimal sensor.

Parameters:

Name	Type	Description	Default
`size`	`tuple`	Physical sensor size (W, H) [mm]. Defaults to `(8.0, 6.0)`.	`(8.0, 6.0)`
`res`	`tuple`	Pixel resolution (W, H). Defaults to `(4000, 3000)`.	`(4000, 3000)`

size `instance-attribute`

size = size

res `instance-attribute`

res = res

isp `instance-attribute`

isp = Sequential(GammaCorrection())

from_config `classmethod`

from_config(sensor_file)

Create a Sensor from a JSON config file.

Parameters:

Name	Type	Description	Default
`sensor_file`		Path to JSON sensor config file.	required

Returns:

Type	Description
	Sensor instance.

to

to(device)

response_curve

response_curve(img_irr)

Apply response curve to the irradiance image to get the raw image.

Default is identity (linear response).

Parameters:

Name	Type	Description	Default
`img_irr`		Irradiance image	required

Returns:

Name	Type	Description
`img_raw`		Raw image

unprocess

unprocess(img)

Inverse ISP: convert sRGB image back to linear RGB.

Parameters:

Name	Type	Description	Default
`img`		Tensor of shape (B, C, H, W), range [0, 1] in sRGB space.	required

Returns:

Name	Type	Description
`img_linear`		Tensor of shape (B, C, H, W), range [0, 1] in linear space.

linrgb2raw

linrgb2raw(img_linear)

Convert linear RGB image to raw sensor space.

For the base Sensor, raw is the linear image itself (identity).

Parameters:

Name	Type	Description	Default
`img_linear`		Tensor of shape (B, C, H, W), range [0, 1].	required

Returns:

Name	Type	Description
`img_raw`		Tensor of shape (B, C, H, W), range [0, 1].

simu_noise

simu_noise(img)

Simulate sensor noise.

Default is identity (no noise).

Parameters:

Name	Type	Description	Default
`img`		Input image	required

Returns:

Name	Type	Description
`img`		Same image unchanged

RGBSensor

Full RGB Bayer sensor with physics-based noise model and invertible ISP. Used by Camera for end-to-end simulation.

from deeplens.sensor import RGBSensor

sensor = RGBSensor(
    size=(36.0, 24.0),        # physical size (W, H) [mm]
    res=(5472, 3648),          # pixel resolution (W, H)
    bit=10,                    # ADC bit depth
    black_level=64,
    bayer_pattern="rggb",
    white_balance=(2.0, 1.0, 1.8),
    gamma_param=2.2,
    iso_base=100,
    read_noise_std=0.5,
    shot_noise_std_alpha=0.4,
)

# Or load from JSON config
sensor = RGBSensor.from_config("imx586.json")

Bayer pattern layout:

R G R G
G B G B
R G R G
G B G B

Noise model:

Shot noise — Poisson photon-counting, scaled by shot_noise_std_alpha
Read noise — Gaussian readout, std = read_noise_std
Higher ISO → amplified noise

Typical pipeline:

import torch

# 1. Render through lens → linear RGB [0, 1]
img_linrgb = lens.render(img, depth=-10000.0)   # (B, 3, H, W)

# 2. Convert to n-bit Bayer
img_bayer = sensor.linrgb2bayer(img_linrgb)      # (B, 1, H, W) in DN

# 3. Add noise + run ISP → sRGB [0, 1]
img_out = sensor.forward(img_bayer, iso=torch.tensor([100]))

# Inverse ISP: sRGB → linear RGB
img_linrgb = sensor.unprocess(img_out)

deeplens.sensor.RGBSensor

RGBSensor(size=(36.0, 24.0), res=(5472, 3648), bit=10, black_level=64, bayer_pattern='rggb', white_balance=(2.0, 1.0, 1.8), color_matrix=None, gamma_param=2.2, iso_base=100, read_noise_std=0.5, shot_noise_std_alpha=0.4, shot_noise_std_beta=0.0, wavelengths=None, red_response=None, green_response=None, blue_response=None)

Bases: Sensor

RGB Bayer-pattern sensor with physics-based noise model and invertible ISP.

Simulates the full image-capture pipeline from linear photon counts to display-ready sRGB:

Spectral integration – optional per-channel spectral response.
Bayer mosaic – pixel-level colour filtering to a single-channel raw image.
Noise – shot noise (signal-dependent Gaussian) + read noise (ISO-independent Gaussian) added to the n-bit raw data.
ISP (forward) – via an :class:~deeplens.sensor.isp_modules.isp.InvertibleISP: black-level correction → white balance → colour matrix → demosaicing → gamma correction.

The ISP is invertible: unprocess() converts sRGB back to linear RGB for training data generation.

Attributes:

Name	Type	Description
`bit`	`int`	ADC bit depth.
`nbit_max`	`int`	Maximum digital number `2**bit - 1`.
`black_level`	`int`	Black level pedestal [DN].
`bayer_pattern`	`str`	Bayer pattern (e.g. `"rggb"`).
`iso_base`	`int`	Base ISO (noise-free reference).
`isp`	`InvertibleISP`	Embedded ISP pipeline.

Parameters:

Name	Type	Description	Default
`size`	`tuple`	Sensor physical size in mm (W, H). Default (36.0, 24.0).	`(36.0, 24.0)`
`res`	`tuple`	Sensor resolution in pixels (W, H). Default (5472, 3648).	`(5472, 3648)`
`bit`	`int`	Bit depth. Default 10.	`10`
`black_level`	`int`	Black level. Default 64.	`64`
`bayer_pattern`	`str`	Bayer pattern e.g. "rggb". Default "rggb".	`'rggb'`
`white_balance`	`tuple`	White balance gains. Default (2.0, 1.0, 1.8).	`(2.0, 1.0, 1.8)`
`color_matrix`	`list or Tensor`	Color correction matrix.	`None`
`gamma_param`	`float`	Gamma correction parameter. Default 2.2.	`2.2`
`iso_base`	`int`	Base ISO. Default 100.	`100`
`read_noise_std`	`float`	Read noise std. Default 0.5.	`0.5`
`shot_noise_std_alpha`	`float`	Shot noise alpha. Default 0.4.	`0.4`
`shot_noise_std_beta`	`float`	Shot noise beta. Default 0.0.	`0.0`
`wavelengths`	`list`	Wavelengths.	`None`
`red_response`	`list`	Red channel spectral response.	`None`
`green_response`	`list`	Green channel spectral response.	`None`
`blue_response`	`list`	Blue channel spectral response.	`None`

bit `instance-attribute`

bit = bit

nbit_max `instance-attribute`

nbit_max = 2 ** bit - 1

black_level `instance-attribute`

black_level = black_level

bayer_pattern `instance-attribute`

bayer_pattern = bayer_pattern

iso_base `instance-attribute`

iso_base = iso_base

readnoise_std `instance-attribute`

readnoise_std = read_noise_std

shotnoise_std_alpha `instance-attribute`

shotnoise_std_alpha = shot_noise_std_alpha

shotnoise_std_beta `instance-attribute`

shotnoise_std_beta = shot_noise_std_beta

wavelengths `instance-attribute`

wavelengths = wavelengths

red_response `instance-attribute`

red_response = tensor(red_response) / green_sum

green_response `instance-attribute`

green_response = tensor(green_response) / green_sum

blue_response `instance-attribute`

blue_response = tensor(blue_response) / green_sum

isp `instance-attribute`

isp = InvertibleISP(bit=bit, black_level=black_level, bayer_pattern=bayer_pattern, white_balance=white_balance, color_matrix=color_matrix, gamma_param=gamma_param)

from_config `classmethod`

from_config(sensor_file)

Create an RGBSensor from a JSON config file.

Parameters:

Name	Type	Description	Default
`sensor_file`		Path to JSON sensor config file.	required

Returns:

Type	Description
	RGBSensor instance.

to

to(device)

response_curve

response_curve(img_spectral)

Apply response curve to the spectral image to get the raw image.

Parameters:

Name	Type	Description	Default
`img_spectral`		Spectral image, shape (B, C, H, W), range [0, 1]	required

Returns:

Name	Type	Description
`img_raw`		Raw image, shape (B, 3, H, W), range [0, 1]

Reference

[1] Spectral Sensitivity Estimation Without a Camera. ICCP 2023. [2] https://github.com/COLOR-Lab-Eilat/Spectral-sensitivity-estimation

unprocess

unprocess(image, in_type='rgb')

Unprocess an image to unbalanced RAW RGB space.

Parameters:

Name	Type	Description	Default
`image`		Tensor of shape (B, 3, H, W), range [0, 1]	required
`in_type`		Input image type, either "rgb" or "linear_rgb"	`'rgb'`

Returns:

Name	Type	Description
`image`		Tensor of shape (B, 3, H, W), range [0, 1] in raw space

linrgb2raw

linrgb2raw(img_linrgb)

Convert linear RGB image to raw Bayer space.

Parameters:

Name	Type	Description	Default
`img_linrgb`		Tensor of shape (B, 3, H, W), range [0, 1]	required

Returns:

Name	Type	Description
`bayer_nbit`		Tensor of shape (B, 1, H, W), range [~black_level, 2**bit - 1]

simu_noise

simu_noise(img_raw, iso)

Simulate sensor noise considering sensor quantization and noise model.

Parameters:

Name	Type	Description	Default
`img_raw`		N-bit clean image, (B, C, H, W), range [0, 2**bit - 1]	required
`iso`		(B,), range [0, 800]	required

Returns:

Name	Type	Description
`img_raw_noise`		N-bit noisy image, (B, C, H, W), range [0, 2**bit - 1]

Reference

[1] "Unprocessing Images for Learned Raw Denoising." [2] https://www.photonstophotos.net/Charts/RN_ADU.htm [3] https://www.photonstophotos.net/Investigations/Measurement_and_Sample_Variation.htm [4] https://www.dpreview.com/forums/thread/4669806

sample_augmentation

sample_augmentation()

Randomly sample a set of augmentation parameters for ISP modules. Used for data augmentation during training.

reset_augmentation

reset_augmentation()

Reset parameters for ISP modules. Used for evaluation.

process2rgb

process2rgb(image, in_type='rggb')

Process an image to a RGB image.

Parameters:

Name	Type	Description	Default
`image`		Tensor of shape (B, 3, H, W), range [0, 1]	required
`in_type`		Input image type, either "rggb" or "bayer"	`'rggb'`

Returns:

Name	Type	Description
`image`		Tensor of shape (B, 3, H, W), range [0, 1]

bayer2rggb

bayer2rggb(bayer_nbit)

Convert RAW bayer image to RAW RGGB image.

Parameters:

Name	Type	Description	Default
`bayer_nbit`		Tensor of shape (B, 1, H, W), range [~black_level, 2**bit - 1]	required

Returns:

Name	Type	Description
`rggb`		Tensor of shape (B, 3, H, W), range [0, 1]

rggb2bayer

rggb2bayer(rggb)

Convert RGGB image to RAW Bayer.

Parameters:

Name	Type	Description	Default
`rggb`		Tensor of shape [4, H/2, W/2] or [B, 4, H/2, W/2], range [0, 1]	required

Returns:

Name	Type	Description
`bayer`		Tensor of shape [1, H, W] or [B, 1, H, W], range [~black_level, 2**bit - 1]

MonoSensor

from deeplens.sensor import MonoSensor

sensor = MonoSensor(
    size=(8.0, 6.0), res=(4000, 3000),
    bit=10, black_level=64,
    iso_base=100, read_noise_std=0.5, shot_noise_std_alpha=0.4,
)
sensor = MonoSensor.from_config("mono_sensor.json")

deeplens.sensor.MonoSensor

MonoSensor(bit=10, black_level=64, size=(8.0, 6.0), res=(4000, 3000), read_noise_std=0.5, shot_noise_std_alpha=0.4, shot_noise_std_beta=0.0, iso_base=100, wavelengths=None, spectral_response=None)

Bases: Sensor

Monochrome sensor with noise simulation and ISP.

Parameters:

Name	Type	Description	Default
`bit`	`int`	Bit depth of the sensor. Default 10.	`10`
`black_level`	`float`	Black level value. Default 64.	`64`
`size`	`tuple`	Sensor physical size in mm (W, H). Default (8.0, 6.0).	`(8.0, 6.0)`
`res`	`tuple`	Sensor resolution in pixels (W, H). Default (4000, 3000).	`(4000, 3000)`
`read_noise_std`	`float`	Read noise standard deviation. Default 0.5.	`0.5`
`shot_noise_std_alpha`	`float`	Shot noise alpha parameter. Default 0.4.	`0.4`
`shot_noise_std_beta`	`float`	Shot noise beta parameter. Default 0.0.	`0.0`
`iso_base`	`int`	Base ISO value. Default 100.	`100`
`wavelengths`	`list`	Wavelengths for spectral response.	`None`
`spectral_response`	`list`	Spectral response values.	`None`

bit `instance-attribute`

bit = bit

nbit_max `instance-attribute`

nbit_max = 2 ** bit - 1

black_level `instance-attribute`

black_level = black_level

iso_base `instance-attribute`

iso_base = iso_base

readnoise_std `instance-attribute`

readnoise_std = read_noise_std

shotnoise_std_alpha `instance-attribute`

shotnoise_std_alpha = shot_noise_std_alpha

shotnoise_std_beta `instance-attribute`

shotnoise_std_beta = shot_noise_std_beta

wavelengths `instance-attribute`

wavelengths = wavelengths

spectral_response `instance-attribute`

spectral_response = response / sum()

isp `instance-attribute`

isp = Sequential(BlackLevelCompensation(bit, black_level), GammaCorrection())

from_config `classmethod`

from_config(sensor_file)

Create a MonoSensor from a JSON config file.

Parameters:

Name	Type	Description	Default
`sensor_file`		Path to JSON sensor config file.	required

Returns:

Type	Description
	MonoSensor instance.

to

to(device)

response_curve

response_curve(img_spectral)

Apply spectral response curve to get a monochrome raw image.

Parameters:

Name	Type	Description	Default
`img_spectral`		Spectral image, (B, N_wavelengths, H, W)	required

Returns:

Name	Type	Description
`img_raw`		Monochrome raw image, (B, 1, H, W)

unprocess

unprocess(img)

Inverse ISP: convert gamma-corrected image back to linear RGB space.

Parameters:

Name	Type	Description	Default
`img`		Tensor of shape (B, C, H, W), range [0, 1] in display space.	required

Returns:

Name	Type	Description
`img_linear`		Tensor of shape (B, C, H, W), range [0, 1] in linear space.

linrgb2raw

linrgb2raw(img_linear)

Convert linear image to n-bit raw digital number.

Applies spectral response (RGB to Mono) and quantization.

Parameters:

Name	Type	Description	Default
`img_linear`		Tensor of shape (B, C, H, W), range [0, 1].	required

Returns:

Name	Type	Description
`img_nbit`		Tensor of shape (B, 1, H, W), range [~black_level, 2**bit - 1].

simu_noise

simu_noise(img_raw, iso)

Simulate sensor noise considering sensor quantization and noise model.

Parameters:

Name	Type	Description	Default
`img_raw`		N-bit clean image, (B, C, H, W), range [0, 2**bit - 1]	required
`iso`		(B,), range [0, 800]	required

Returns:

Name	Type	Description
`img_raw_noise`		N-bit noisy image, (B, C, H, W), range [0, 2**bit - 1]

Reference

[1] "Unprocessing Images for Learned Raw Denoising." [2] https://www.photonstophotos.net/Charts/RN_ADU.htm [3] https://www.photonstophotos.net/Investigations/Measurement_and_Sample_Variation.htm [4] https://www.dpreview.com/forums/thread/4669806

EventSensor

from deeplens.sensor import EventSensor

sensor = EventSensor(
    size=(8.0, 6.0), res=(640, 480),
    threshold_pos=0.2, threshold_neg=0.2, sigma_threshold=0.03,
)
sensor = EventSensor.from_config("event_sensor.json")

deeplens.sensor.EventSensor

EventSensor(size=(8.0, 6.0), res=(4000, 3000), threshold_pos=0.2, threshold_neg=0.2, sigma_threshold=0.03, cutoff_hz=0.0, leak_rate_hz=0.0, shot_noise_rate_hz=0.0, eps=1e-07)

Bases: Sensor

Event sensor (Dynamic Vision Sensor) simulation.

An event camera independently measures per-pixel log-intensity changes and fires an event whenever the change exceeds a contrast threshold:

e_k = (x_k, y_k, t_k, p_k)

where p_k ∈ {+1, -1} is the polarity. This class produces dense event count maps of shape (B, 2, H, W) where channel 0 counts positive events and channel 1 counts negative events.

Parameters:

Name	Description	Default
`size`	Physical sensor size in mm, (width, height).	`(8.0, 6.0)`
`res`	Sensor resolution in pixels, (width, height).	`(4000, 3000)`
`threshold_pos`	Positive contrast threshold (C+). Default 0.2.	`0.2`
`threshold_neg`	Negative contrast threshold (C-). Default 0.2.	`0.2`
`sigma_threshold`	Std-dev of threshold noise. Set to 0 for deterministic operation. Default 0.03.	`0.03`
`cutoff_hz`	High-pass temporal filter cutoff frequency in Hz. Set to 0 to disable. Default 0.	`0.0`
`leak_rate_hz`	Leak event rate in Hz. Simulates spontaneous background events. Set to 0 to disable. Default 0.	`0.0`
`shot_noise_rate_hz`	Shot noise event rate in Hz. Simulates noise events from dark current. Set to 0 to disable. Default 0.	`0.0`
`eps`	Small constant added before log to avoid log(0). Default 1e-7.	`1e-07`

threshold_pos `instance-attribute`

threshold_pos = threshold_pos

threshold_neg `instance-attribute`

threshold_neg = threshold_neg

sigma_threshold `instance-attribute`

sigma_threshold = sigma_threshold

cutoff_hz `instance-attribute`

cutoff_hz = cutoff_hz

leak_rate_hz `instance-attribute`

leak_rate_hz = leak_rate_hz

shot_noise_rate_hz `instance-attribute`

shot_noise_rate_hz = shot_noise_rate_hz

eps `instance-attribute`

eps = eps

from_config `classmethod`

from_config(sensor_file)

Create an EventSensor from a JSON config file.

Parameters:

Name	Type	Description	Default
`sensor_file`		Path to JSON sensor config file.	required

Returns:

Type	Description
	EventSensor instance.

_to_gray

_to_gray(img)

Convert to single-channel grayscale if needed.

Parameters:

Name	Type	Description	Default
`img`		Tensor (B, C, H, W), range [0, 1].	required

Returns:

Type	Description
	Tensor (B, 1, H, W).

_log_intensity

_log_intensity(img)

Compute log intensity.

Parameters:

Name	Type	Description	Default
`img`		Tensor (B, 1, H, W), range [0, 1].	required

Returns:

Type	Description
	Log intensity tensor (B, 1, H, W).

forward

forward(I_t, I_t_1, dt=None)

Convert two consecutive frames into a dense event count map.

Each pixel independently computes the change in log intensity and fires integer-count positive or negative events when the change exceeds the contrast threshold.

Parameters:

Name	Description	Default
`I_t`	Current frame, tensor (B, C, H, W), range [0, 1].	required
`I_t_1`	Previous frame, tensor (B, C, H, W), range [0, 1].	required
`dt`	Time interval between frames in seconds. Required when `cutoff_hz`, `leak_rate_hz`, or `shot_noise_rate_hz` is non-zero. Default None.	`None`

Returns:

Name	Type	Description
`events`		Tensor (B, 2, H, W). Channel 0 = positive event counts, channel 1 = negative event counts.

forward_video

forward_video(frames, dt=None)

Simulate event sensor output from a video sequence.

Iterates over consecutive frame pairs and generates event count maps for each transition.

Parameters:

Name	Type	Description	Default
`frames`		Tensor (B, T, C, H, W), range [0, 1].	required
`dt`		Time interval between frames in seconds. Default None.	`None`

Returns:

Name	Type	Description
`events`		Tensor (B, T-1, 2, H, W). Event count maps for each consecutive frame pair.

events_to_voxel_grid

events_to_voxel_grid(events, num_bins=5)

Convert event count map to a temporal voxel grid representation.

Distributes events across num_bins temporal bins using linear interpolation. This is a common input representation for event- based neural networks.

Parameters:

Name	Type	Description	Default
`events`		Tensor (B, 2, H, W) — single-pair event counts.	required
`num_bins`		Number of temporal bins. Default 5.	`5`

Returns:

Name	Type	Description
`voxel`		Tensor (B, num_bins, H, W).

events_to_timestamp_image

events_to_timestamp_image(events)

Convert event count map to a 2-channel timestamp-like image.

Creates a representation where non-zero pixels indicate event activity. Channel 0 stores positive event magnitude, channel 1 stores negative event magnitude, both normalised to [0, 1].

Parameters:

Name	Type	Description	Default
`events`		Tensor (B, 2, H, W).	required

Returns:

Name	Type	Description
`ts_img`		Tensor (B, 2, H, W), range [0, 1].

ISP Modules

RGBSensor embeds an InvertibleISP that implements both the forward (RAW → sRGB) and inverse (sRGB → linear RAW) pipelines, enabling "unprocessing" of real images for realistic training data.

InvertibleISP

deeplens.sensor.isp_modules.isp.InvertibleISP

InvertibleISP(bit=10, black_level=64, bayer_pattern='rggb', white_balance=(2.0, 1.0, 1.8), color_matrix=None, gamma_param=2.2)

Bases: Module

Invertible and differentiable Bayer-sRGB ISP pipeline.

Rerference

[1] Architectural Analysis of a Baseline ISP Pipeline. https://link.springer.com/chapter/10.1007/978-94-017-9987-4_2. (page 23, 50)

bit `instance-attribute`

bit = bit

black_level `instance-attribute`

black_level = black_level

bayer_pattern `instance-attribute`

bayer_pattern = bayer_pattern

white_balance `instance-attribute`

white_balance = white_balance

color_matrix `instance-attribute`

color_matrix = color_matrix

gamma_param `instance-attribute`

gamma_param = gamma_param

blc `instance-attribute`

blc = BlackLevelCompensation(bit=bit, black_level=black_level)

demosaic `instance-attribute`

demosaic = Demosaic(bayer_pattern=bayer_pattern, method='malvar')

awb `instance-attribute`

awb = AutoWhiteBalance(awb_method='manual', white_balance=white_balance)

ccm `instance-attribute`

ccm = ColorCorrectionMatrix(ccm_matrix=color_matrix)

gamma `instance-attribute`

gamma = GammaCorrection(gamma_param=gamma_param)

isp `instance-attribute`

isp = Sequential(blc, demosaic, awb, ccm, gamma)

forward

forward(bayer_nbit)

A basic differentiable and invertible ISP pipeline.

Parameters:

Name	Type	Description	Default
`bayer_Nbit`		Input tensor of shape [B, 1, H, W], data range [~black_level, 2^bit-1].	required

Returns:

Name	Type	Description
`rgb`		Output tensor of shape [B, 3, H, W], data range [0, 1].

reverse

reverse(img)

Inverse ISP.

Parameters:

Name	Type	Description	Default
`img`		Input tensor of shape [B, 3, H, W], data range [0, 1].	required

Returns:

Name	Type	Description
`bayer_Nbit`		Output tensor of shape [B, 1, H, W], data range [~black_level, 2^bit-1].

Individual ISP Stages

The ISP pipeline is composed of modular stages that can be used independently:

Black Level Correction

from deeplens.sensor.isp_modules import BlackLevel
corrected = BlackLevel(level=64)(raw_image)

White Balance

from deeplens.sensor.isp_modules import WhiteBalance
balanced = WhiteBalance(method='gray_world', gains=[1.5, 1.0, 1.8])(raw_image)

Demosaicing

Method	Description
`bilinear`	Fast, simple interpolation
`malvar`	Edge-aware (recommended for speed)
`menon`	High-quality, edge-directed
`ahd`	Adaptive homogeneity-directed

from deeplens.sensor.isp_modules import Demosaic
rgb = Demosaic(method='malvar')(raw_image)

Colour Correction Matrix

from deeplens.sensor.isp_modules import ColorMatrix
corrected = ColorMatrix(matrix=torch.tensor([[1.5,-0.3,-0.2],[-0.2,1.3,-0.1],[-0.1,-0.4,1.5]]))(rgb)

Gamma Correction

from deeplens.sensor.isp_modules import GammaCorrection
out = GammaCorrection(gamma=2.2)(linear_rgb)

Custom ISP Pipeline

import torch
from deeplens.sensor.isp_modules import BlackLevel, WhiteBalance, Demosaic, ColorMatrix, GammaCorrection

class CustomISP(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.bl  = BlackLevel(level=64)
        self.wb  = WhiteBalance(method='gray_world')
        self.dm  = Demosaic(method='malvar')
        self.ccm = ColorMatrix()
        self.gm  = GammaCorrection(gamma=2.2)
    def forward(self, raw):
        return self.gm(self.ccm(self.dm(self.wb(self.bl(raw)))))

Sensor Formats

Common sensor sizes for reference:

Format	Width (mm)	Height (mm)	Diagonal (mm)
Full Frame	36.0	24.0	43.3
APS-C (Canon)	22.2	14.8	26.7
APS-C (Nikon)	23.5	15.6	28.2
Micro 4/3	17.3	13.0	21.6
1"	13.2	8.8	15.9
1/2.3"	6.2	4.6	7.7

Best Practices

Pixel size: larger → better SNR; smaller → higher resolution
Bit depth: 12–14 bits is sufficient for most applications
ISP order matters: apply corrections in sequence — black level → WB → demosaic → CCM → gamma
Validate noise: compare shot/read noise statistics against real sensor datasheet
Invertible ISP: use sensor.unprocess() to generate realistic RAW training inputs from sRGB images

Sensor API Reference

Sensor Types

Sensor (base)

deeplens.sensor.Sensor

size instance-attribute

res instance-attribute

isp instance-attribute

from_config classmethod

to

response_curve

unprocess

linrgb2raw

simu_noise

RGBSensor

deeplens.sensor.RGBSensor

bit instance-attribute

nbit_max instance-attribute

black_level instance-attribute

bayer_pattern instance-attribute

iso_base instance-attribute

readnoise_std instance-attribute

shotnoise_std_alpha instance-attribute

shotnoise_std_beta instance-attribute

wavelengths instance-attribute

red_response instance-attribute

green_response instance-attribute

blue_response instance-attribute

isp instance-attribute

from_config classmethod

to

response_curve

unprocess

linrgb2raw

simu_noise

sample_augmentation

reset_augmentation

process2rgb

bayer2rggb

rggb2bayer

MonoSensor

deeplens.sensor.MonoSensor

bit instance-attribute

nbit_max instance-attribute

black_level instance-attribute

iso_base instance-attribute

readnoise_std instance-attribute

shotnoise_std_alpha instance-attribute

shotnoise_std_beta instance-attribute

wavelengths instance-attribute

spectral_response instance-attribute

isp instance-attribute

from_config classmethod

to

response_curve

unprocess

linrgb2raw

simu_noise

EventSensor

deeplens.sensor.EventSensor

threshold_pos instance-attribute

threshold_neg instance-attribute

sigma_threshold instance-attribute

cutoff_hz instance-attribute

leak_rate_hz instance-attribute

shot_noise_rate_hz instance-attribute

eps instance-attribute

from_config classmethod

_to_gray

_log_intensity

forward

forward_video

events_to_voxel_grid

events_to_timestamp_image

ISP Modules

InvertibleISP

deeplens.sensor.isp_modules.isp.InvertibleISP

bit instance-attribute

black_level instance-attribute

bayer_pattern instance-attribute

white_balance instance-attribute

size `instance-attribute`

res `instance-attribute`

isp `instance-attribute`

from_config `classmethod`

bit `instance-attribute`

nbit_max `instance-attribute`

black_level `instance-attribute`

bayer_pattern `instance-attribute`

iso_base `instance-attribute`

readnoise_std `instance-attribute`

shotnoise_std_alpha `instance-attribute`

shotnoise_std_beta `instance-attribute`

wavelengths `instance-attribute`

red_response `instance-attribute`

green_response `instance-attribute`

blue_response `instance-attribute`

isp `instance-attribute`

from_config `classmethod`

bit `instance-attribute`

nbit_max `instance-attribute`

black_level `instance-attribute`

iso_base `instance-attribute`

readnoise_std `instance-attribute`

shotnoise_std_alpha `instance-attribute`

shotnoise_std_beta `instance-attribute`

wavelengths `instance-attribute`

spectral_response `instance-attribute`

isp `instance-attribute`

from_config `classmethod`

threshold_pos `instance-attribute`

threshold_neg `instance-attribute`

sigma_threshold `instance-attribute`

cutoff_hz `instance-attribute`

leak_rate_hz `instance-attribute`

shot_noise_rate_hz `instance-attribute`

eps `instance-attribute`

from_config `classmethod`

bit `instance-attribute`

black_level `instance-attribute`

bayer_pattern `instance-attribute`

white_balance `instance-attribute`

color_matrix `instance-attribute`

gamma_param `instance-attribute`

blc `instance-attribute`

demosaic `instance-attribute`

awb `instance-attribute`

ccm `instance-attribute`

gamma `instance-attribute`

isp `instance-attribute`