Tutorials

This section provides comprehensive tutorials for using DeepLens.

Tutorial Series

The DeepLens repository includes several tutorial scripts that cover different aspects of the library:

Script

Description

0_hello_deeplens.py

Introduction to DeepLens basics: loading lenses, ray tracing, PSF calculation, and image rendering

1_end2end_lens_design.py

End-to-end lens optimization with vision tasks

2_autolens_rms.py

Automated lens design using curriculum learning

3_psf_net.py

Training neural surrogate models for fast PSF prediction

4_tasklens_img_classi.py

Task-specific lens design for image classification

5_pupil_field.py

Pupil and field analysis

6_hybridlens_design.py

Hybrid refractive-diffractive lens design

7_comp_photography.py

Computational photography applications

Tutorial 1: Loading and Analyzing Lenses

Loading Lens Files

DeepLens supports multiple lens file formats:

from deeplens import GeoLens

# Load from JSON format
lens = GeoLens(filename='./datasets/lenses/camera/ef50mm_f1.8.json')

# Load from Zemax format (.zmx)
lens = GeoLens(filename='./datasets/lenses/zemax_double_gaussian.zmx')

Lens Properties

Access key lens properties:

import math
print(f"Focal length: {lens.foclen:.2f} mm")
print(f"F-number: {lens.fnum:.2f}")
# Compute entrance pupil diameter from pupil parameters
_, entr_r = lens.get_entrance_pupil()
print(f"Entrance pupil diameter: {2 * entr_r:.2f} mm")
print(f"Horizontal FoV: {math.degrees(lens.hfov):.1f} deg")
print(f"Number of surfaces: {len(lens.surfaces)}")

Visualization

Visualize the lens system:

# 2D layout with ray tracing
lens.draw_layout(filename="layout.png", depth=float("inf"))

# 3D visualization (saved to directory)
lens.draw_lens_3d(save_dir="./vis3d")

Tutorial 2: Ray Tracing

Creating Ray Bundles

Different ways to create rays:

# 2D parallel rays (for layout/on-axis analysis)
ray = lens.sample_parallel_2D(fov=0.0, num_rays=11, plane="sagittal")

# 3D parallel rays (for geometric analysis)
ray = lens.sample_parallel(fov_x=0.0, fov_y=0.0, num_rays=1024)

# Point source rays (object at 1 m; object-space depths are negative)
ray = lens.sample_point_source(fov_x=0.0, fov_y=0.0, depth=-1000.0, num_rays=2048)

# Rays from absolute 3D points in object space
ray = lens.sample_from_points(points=[[0.0, 0.0, -10000.0]], num_rays=2048)

Ray Tracing

Trace rays through the lens:

# Trace (returns output rays and optional intersection records)
ray_out, _ = lens.trace(ray)

# Check which rays reached the sensor
num_valid = int(ray_out.valid.sum().item())
num_total = ray_out.valid.numel()
print(f"Valid rays: {num_valid} / {num_total}")

Tutorial 3: Point Spread Function (PSF)

Basic PSF Calculation

import torch
import matplotlib.pyplot as plt

# Single-point PSF at 10 m, centered field (normalized x=y=0)
psf = lens.psf(points=torch.tensor([0.0, 0.0, -10000.0]), ks=64, spp=2048)

# Visualize
plt.imshow(psf.cpu(), cmap="inferno")
plt.axis("off")
plt.show()

PSF Across the Field

Calculate PSF map across different field positions:

# Compute and save PSF map across field
psf_map = lens.psf_map(depth=-10000.0, grid=(7, 7), ks=64, spp=1024)
lens.draw_psf_map(grid=(7, 7), ks=64, depth=-10000.0, save_name="psf_map.png")

Depth-Varying PSF

Analyze defocus effects:

import matplotlib.pyplot as plt

depths = [-5000, -10000, -20000]

fig, axes = plt.subplots(1, len(depths), figsize=(15, 3))
for i, depth in enumerate(depths):
    psf = lens.psf(points=torch.tensor([0.0, 0.0, depth]), ks=64, spp=1024)
    axes[i].imshow(psf.cpu(), cmap="inferno")
    axes[i].set_title(f'{abs(depth)} mm')
    axes[i].axis('off')
plt.show()

Tutorial 4: Image Rendering

Basic Image Rendering

from PIL import Image
import torchvision.transforms as transforms
from torchvision.utils import save_image

# Load image
img = Image.open('./datasets/bird.png')
img_tensor = transforms.ToTensor()(img).unsqueeze(0).cuda()

# Match sensor resolution to image for full-frame rendering
lens.set_sensor_res(sensor_res=(img_tensor.shape[-1], img_tensor.shape[-2]))

# Render through lens (ray tracing)
img_rendered = lens.render(img_tensor, depth=-10000.0, method="ray_tracing", spp=256)

# Save result
save_image(img_rendered, 'output.png')

Depth-Aware Rendering

Render scenes with depth variation:

# Load RGB and depth
img_rgb = Image.open('./datasets/edof/rgb.png')
img_depth = Image.open('./datasets/edof/depth.png')

rgb_tensor = transforms.ToTensor()(img_rgb).unsqueeze(0).cuda()
depth_map = transforms.ToTensor()(img_depth).unsqueeze(0).cuda()

# Scale depth to millimeters and use negative sign for object space
depth_map = - (depth_map * 5000 + 500)  # 500mm to 5500mm -> -[500, 5500] mm

# Render with depth using PSF interpolation
img_rendered = lens.render_rgbd(rgb_tensor, depth_map, method="psf_map", psf_grid=(10, 10), psf_ks=64)

save_image(img_rendered, 'depth_rendered.png')

Tutorial 5: Lens Optimization

Basic Optimization Setup

# Get optimizer for lens parameters
optimizer = lens.get_optimizer(lrs=[1e-4, 1e-4, 1e-2, 1e-4], decay=0.01)

Optimization Loop

for iteration in range(1000):
    optimizer.zero_grad()

    # RMS spot error across field (geometric objective)
    loss = lens.loss_rms(num_grid=9, depth=-10000.0, num_rays=2048)

    # Regularization for physical feasibility (spacing, angles, thickness)
    loss_reg, _ = lens.loss_reg()
    loss = loss + 0.05 * loss_reg

    # Backpropagation
    loss.backward()
    optimizer.step()

    if iteration % 100 == 0:
        print(f"Iteration {iteration}, Loss: {loss.item():.6f}")

Tutorial 6: Using Neural Surrogates

PSFNetLens

Fast PSF prediction using neural networks:

import torch
from deeplens import PSFNetLens

# Initialize and load pretrained PSF network
lens = PSFNetLens(
    lens_path='./datasets/lenses/camera/ef50mm_f1.8.json',
    sensor_res=(3000, 3000)
)
lens.load_net('./ckpts/psfnet/PSFNet_ef50mm_f1.8_ps10um.pth')

# Fast PSF calculation (RGB PSF)
psf_rgb = lens.psf_rgb(points=torch.tensor([[0.0, 0.0, -10000.0]]), ks=64)

# Rendering via PSF map using the surrogate
img_rendered = lens.render(img_tensor, depth=-10000.0, method='psf_map')

Training a Surrogate Model

See 3_psf_net.py for a complete example of training your own PSF network:

python 3_psf_net.py

Tutorial 7: Camera Systems

Creating a Camera

from deeplens import Camera

camera = Camera(
    lens_file='./datasets/lenses/camera/ef50mm_f1.8.json',
    sensor_file='./datasets/sensors/canon_r6.json',
    device='cuda'
)

Image Signal Processing

Apply ISP pipeline:

from deeplens.sensor.isp_modules.isp import InvertibleISP

# Create ISP
isp = InvertibleISP(bit=10, black_level=64, bayer_pattern='rggb')

# Process RAW Bayer data to RGB
# raw_bayer = ...  # shape (B, 1, H, W)
# rgb = isp(raw_bayer)

Configuration Files

DeepLens supports YAML configuration files for reproducible experiments:

# configs/1_end2end_lens_design.yml
lens:
  filename: './datasets/lenses/camera/ef50mm_f1.8.json'
  sensor_res: [256, 256]

optimization:
  learning_rate: 0.01
  iterations: 1000
  loss_type: 'rms_spot'

constraints:
  min_thickness: 0.5
  max_thickness: 20.0

Load configuration:

import yaml

with open('configs/1_end2end_lens_design.yml', 'r') as f:
    config = yaml.safe_load(f)

lens = GeoLens(**config['lens'])

Next Steps