Lens API Reference¶

This section documents the lens classes and their methods.

Base Lens Class¶

class Lens(dtype=torch.float32, device=None)¶

Base class for all lens systems in DeepLens.

Parameters:

dtype – Data type for computations (default: torch.float32)
device – Device to use (‘cuda’ or ‘cpu’)

Note

Prefer keyword arguments when constructing lenses; positional arguments follow (dtype, device).

psf(points, wvln=0.587, ks=64, **kwargs)¶

Compute monochrome point PSF. This function should be differentiable.

Parameters:

points (torch.Tensor) – Point positions, shape [N, 3] or [3], normalized xy in [-1, 1], z < 0
wvln (float) – Wavelength in micrometers
ks (int) – Kernel size

Returns:

PSF tensor [ks, ks] or [N, ks, ks]

Return type:

torch.Tensor

render(img_obj, depth=-20000.0, method='psf_patch', **kwargs)¶

Differentiable image simulation through the lens.

Parameters:

img_obj (torch.Tensor) – Input image tensor [B, C, H, W]
depth (float) – Object depth in mm
method (str) – Rendering method - ‘psf_map’ or ‘psf_patch’
kwargs – Additional method-specific arguments (psf_grid, psf_ks, patch_center)

Returns:

Rendered image tensor [B, C, H, W]

Return type:

torch.Tensor

render_rgbd(img_obj, depth_map, method='psf_patch', **kwargs)¶

Render RGBD image using depth map.

Parameters:

img_obj (torch.Tensor) – Input image tensor [B, C, H, W]
depth_map (torch.Tensor) – Depth map tensor [B, 1, H, W]
method (str) – Rendering method - ‘psf_map’, ‘psf_patch’, or ‘psf_pixel’
kwargs (dict) – Additional arguments (interp_mode, depth_min, depth_max, num_depth, psf_grid, psf_ks, patch_center)

Returns:

Rendered image tensor [B, C, H, W]

Return type:

torch.Tensor

set_sensor(sensor_size, sensor_res)¶

Set sensor dimensions.

Parameters:

sensor_size (tuple) – Physical sensor size (W, H) in mm
sensor_res (tuple) – Sensor resolution (W, H) in pixels

set_sensor_res(sensor_res)¶

Set sensor resolution while keeping sensor radius unchanged.

Parameters:: sensor_res (tuple) – Sensor resolution (W, H) in pixels

to(device)¶

Move lens to specified device.

Parameters:: device – ‘cuda’ or ‘cpu’
Returns:: self

GeoLens¶

class GeoLens(filename=None, device=None, dtype=torch.float32)¶

Geometric lens system using ray tracing.

Parameters:

filename – Path to lens file (.json, .zmx)
device – Device to use
dtype – Data type

Note

Sensor size and resolution are read from the lens file. If not provided, defaults of 8mm x 8mm and 2000x2000 pixels will be used.

Attributes:

surfaces¶: List of optical surfaces

materials¶: List of optical materials

foclen¶: Focal length in mm (float)

fnum¶: F-number (float)

enpd¶: Entrance pupil diameter in mm (float)

hfov¶: Horizontal field of view in degrees (float)

Methods:

read_lens(filename)¶

Load lens from file.

Parameters:: filename – Path to .json or .zmx file

write_lens_json(filename)¶

Save lens to JSON file.

Parameters:: filename – Output file path

trace(ray)¶

Trace rays through the lens.

Parameters:: ray – Input Ray object
Returns:: Output Ray object

sample_parallel_2D(fov=0.0, num_rays=7, wvln=0.587, plane='meridional', entrance_pupil=True, depth=0.0)¶

Sample 2D parallel rays for layout visualization.

Parameters:

fov (float) – Field angle in degrees
num_rays (int) – Number of rays
wvln (float) – Wavelength in micrometers
plane (str) – ‘meridional’ or ‘sagittal’
entrance_pupil (bool) – Use entrance pupil
depth (float) – Sampling depth

Returns:

Ray object with shape [num_rays, 3]

Return type:

Ray

sample_point_source(fov_x=[0.0], fov_y=[0.0], depth=-20000.0, num_rays=16384, wvln=0.587, entrance_pupil=True, scale_pupil=1.0)¶

Sample point source rays from object space with given field angles.

Parameters:

fov_x (float or list) – Field angle(s) in x direction in degrees
fov_y (float or list) – Field angle(s) in y direction in degrees
depth (float) – Object depth in mm
num_rays (int) – Rays per field point
wvln (float) – Wavelength in micrometers
entrance_pupil (bool) – If True, use entrance pupil
scale_pupil (float) – Scale factor for pupil radius

Returns:

Ray object with shape [len(fov_y), len(fov_x), num_rays, 3]

Return type:

Ray

sample_from_points(points=[[0.0, 0.0, -10000.0]], num_rays=16384, wvln=0.587, scale_pupil=1.0)¶

Sample rays from point sources at absolute 3D coordinates.

Parameters:

points (list or torch.Tensor) – Point source positions in shape [3], [N, 3], or [Nx, Ny, 3]
num_rays (int) – Rays per point
wvln (float) – Wavelength in micrometers
scale_pupil (float) – Scale factor for pupil radius

Returns:

Sampled rays with shape [*points.shape[:-1], num_rays, 3]

Return type:

Ray

get_optimizer_params(lrs=[1e-4, 1e-4, 1e-2, 1e-4], decay=0.01, optim_mat=False, optim_surf_range=None)¶

Get optimizer parameter groups for lens optimization.

Parameters:

lrs – Learning rates for [thickness, curvature, conic, aspheric coefficients]
decay – Decay factor for higher-order aspheric coefficients
optim_mat – Whether to optimize material properties
optim_surf_range – Optional surface indices to optimize

Example:

params = lens.get_optimizer_params(lrs=[1e-4, 1e-4, 1e-2, 1e-4])
optimizer = torch.optim.Adam(params)

calc_foclen()¶

Calculate focal length.

Returns:: Focal length in mm

calc_pupil()¶: Calculate entrance pupil position and diameter.

calc_fov()¶: Calculate field of view.

loss_constraint()¶

Calculate constraint violations for optimization.

Returns:: Constraint loss (scalar tensor)

plot_setup2D(M=10, plot_rays=True)¶

Plot 2D cross-section of lens.

Parameters:

M – Number of rays to plot
plot_rays – Whether to show ray traces

plot_setup3D()¶: Plot 3D visualization of lens.

plot_psf(psf, figsize=(10, 8))¶

Visualize PSF.

Parameters:

psf – PSF tensor [C, H, W]
figsize – Figure size

plot_psf_map(psf_map, figsize=(15, 10))¶

Visualize PSF across field.

Parameters:

psf_map – PSF map tensor
figsize – Figure size

analysis_rms_spot()¶

Analyze RMS spot size across field.

Returns:: RMS spot size map

analysis_distortion()¶

Analyze geometric distortion.

Returns:: Distortion map

analysis_mtf(frequency=50)¶

Analyze Modulation Transfer Function.

Parameters:: frequency – Spatial frequency in lp/mm
Returns:: MTF map

PSFNetLens¶

class PSFNetLens(lens_path, in_chan=3, psf_chan=3, model_name='mlp_conv', kernel_size=64)¶

Neural surrogate lens model that represents the PSF using a neural network.

Parameters:

lens_path (str) – Path to the lens JSON file
in_chan (int) – Number of input channels (fov, depth, foc_dist)
psf_chan (int) – Number of output PSF channels (RGB)
model_name (str) – Network architecture (‘mlp’ or ‘mlpconv’)
kernel_size (int) – PSF kernel size

Note

Sensor size and resolution are read from the lens file. If not provided, GeoLens defaults will be used.

Attributes:

lens¶: Embedded GeoLens object

psfnet¶: Neural network for PSF prediction

pixel_size¶: Pixel size in mm (float)

Methods:

psf_rgb(points, ks=64)¶

Fast RGB PSF prediction using neural network.

Parameters:

points (torch.Tensor) – Point positions [N, 3], normalized xy in [-1, 1], z is depth in mm
ks (int) – Kernel size

Returns:

PSF tensor [N, 3, ks, ks]

Return type:

torch.Tensor

render_rgbd(img, depth, foc_dist, ks=64, high_res=False)¶

Render image with depth map using per-pixel PSF convolution.

Parameters:

img (torch.Tensor) – Input image [1, C, H, W]
depth (torch.Tensor) – Depth map [1, H, W] in mm
foc_dist (float) – Focus distance in mm
ks (int) – PSF kernel size
high_res (bool) – Use high resolution rendering

Returns:

Rendered image [1, C, H, W]

Return type:

torch.Tensor

refocus(foc_dist)¶

Refocus the lens to a given focus distance.

Parameters:: foc_dist (float) – Focus distance in mm

load_net(net_path)¶

Load pretrained network weights.

Parameters:: net_path (str) – Path to network checkpoint

train_psfnet(iters=100000, bs=128, lr=5e-5, evaluate_every=500, spp=16384, concentration_factor=2.0, result_dir='./results/psfnet')¶

Train the PSF surrogate network.

Parameters:

iters (int) – Number of training iterations
bs (int) – Batch size
lr (float) – Learning rate
evaluate_every (int) – Evaluation interval
spp (int) – Samples per pixel for ray tracing
concentration_factor (float) – Concentration factor for training data sampling
result_dir (str) – Directory to save results

HybridLens¶

class HybridLens(filename=None, device=None, dtype=torch.float64)¶

Hybrid refractive-diffractive lens using a differentiable ray-wave model.

Parameters:

filename (str or None) – Path to hybrid-lens JSON file. If None, create empty hybrid lens
device (str or None) – Computing device (‘cuda’ or ‘cpu’). If None, auto-selects
dtype (torch.dtype) – Data type for computations (default: torch.float64)

Note

Sensor size and resolution are read from the lens file (GeoLens section). If not provided, defaults of 8mm x 8mm and 2000x2000 pixels will be used.

Methods:

psf(points=[0.0, 0.0, -10000.0], ks=PSF_KS, wvln=DEFAULT_WAVE, spp=SPP_COHERENT)¶

Single point monochromatic PSF using ray-wave model.

Parameters:

points (list or torch.Tensor) – Point source position [x, y, z] (normalized x,y in [-1, 1], z < 0)
ks (int or None) – Output PSF kernel size
wvln (float) – Wavelength in micrometers
spp (int) – Rays per point for coherent tracing (>= 1e6 recommended)

Returns:

Normalized PSF patch [ks, ks]

Return type:

torch.Tensor

draw_layout(save_name='./DOELens.png', depth=-10000.0, ax=None, fig=None)¶

Draw the hybrid system layout with ray-tracing and wave-propagation.

Parameters:

save_name (str) – Output figure path
depth (float) – Object depth for ray bundles (mm)
ax (matplotlib.axes.Axes or None) – Optional matplotlib axis
fig (matplotlib.figure.Figure or None) – Optional matplotlib figure

ParaxialLens¶

class ParaxialLens(foclen, fnum, sensor_size=None, sensor_res=None, device='cpu')¶

Paraxial (thin lens) model.

Parameters:

foclen – Focal length in mm
fnum – F-number
sensor_size – Sensor size (W, H) in mm. If None, defaults to (8.0, 8.0)
sensor_res – Sensor resolution (W, H) in pixels. If None, defaults to (2000, 2000)
device – Device to use

Methods:

render(img, depth, focus_depth=None)¶

Render with defocus blur.

Parameters:

img – Input image
depth – Object distance in mm
focus_depth – Focus distance (default: depth)

Returns:

Blurred image

Camera¶

class Camera(lens, sensor=None, isp=None, device='cuda')¶

Complete camera system.

Parameters:

lens – Lens object
sensor – Sensor object
isp – ISP pipeline object
device – Device to use

Attributes:

lens¶: Lens system

sensor¶: Image sensor

isp¶: Image signal processor

Methods:

capture(scene, depth, exposure_time=0.01, iso=100, auto_focus=False)¶

Capture image end-to-end.

Parameters:

scene – Scene image tensor
depth – Object distance or depth map
exposure_time – Exposure time in seconds
iso – ISO sensitivity
auto_focus – Enable auto-focus

Returns:

Processed image

capture_raw(scene, depth, exposure_time=0.01, iso=100)¶

Capture raw sensor data.

Parameters:

scene – Scene image
depth – Object distance
exposure_time – Exposure time
iso – ISO setting

Returns:

Raw sensor data

set_focus(focus_depth)¶

Set focus distance.

Parameters:: focus_depth – Focus distance in mm

auto_focus(scene, depth)¶

Automatic focus adjustment.

Parameters:

scene – Scene for focus detection
depth – Depth information

Examples¶

Basic Usage¶

from deeplens import GeoLens

# Create lens
lens = GeoLens(
    filename='./datasets/lenses/camera/ef50mm_f1.8.json',
    device='cuda'
)

# Calculate PSF
psf = lens.psf(points=[0.0, 0.0, -1000.0], spp=2048)

# Render image
img_rendered = lens.render(img, depth=-1000)

Lens Optimization¶

import torch.optim as optim
from deeplens.optics import SpotLoss

# Setup optimization
params = lens.get_optimizer_params(lrs=[1e-4, 1e-4, 1e-2, 1e-4])
optimizer = optim.Adam(params)
loss_fn = SpotLoss()

# Optimization loop
for i in range(1000):
    optimizer.zero_grad()
    ray = lens.sample_point_source(
        depth=-1e4,
        num_rays=256,  # 256 vs 16384 for faster iterations (lower sampling accuracy per step)
    )
    ray_out, _ = lens.trace(ray)
    loss = loss_fn(ray_out) + lens.loss_constraint()
    loss.backward()
    optimizer.step()

Fast PSF Prediction¶

import torch
from deeplens import PSFNetLens

# Load pre-trained model
lens = PSFNetLens(lens_path='./datasets/lenses/camera/ef50mm_f1.8.json')
lens.load_net('./ckpts/psfnet/PSFNet_ef50mm_f1.8_ps10um.pth')

# Fast PSF calculation
psf = lens.psf_rgb(points=torch.tensor([[0.0, 0.5, -1000.0]]), ks=64).squeeze(0)

# 100x faster than ray tracing!

Camera System¶

from deeplens import Camera, GeoLens
from deeplens.sensor import RGBSensor, ISP

# Create camera
camera = Camera(
    lens=GeoLens(filename='lens.json'),
    sensor=RGBSensor(resolution=(1920, 1080)),
    isp=ISP()
)

# Capture image
img = camera.capture(scene, depth=1000, exposure_time=0.01)

DeepLens

Navigation

Related Topics

Lens API Reference¶

Base Lens Class¶

GeoLens¶

PSFNetLens¶

HybridLens¶

ParaxialLens¶

Camera¶

Examples¶

Basic Usage¶

Lens Optimization¶

Fast PSF Prediction¶

Camera System¶

See Also¶