Imported from hiyenwong/ai_collection (collection/skills/spikerestormer-unified-event-reasoning/SKILL.md). Install upstream with npx skills add hiyenwong/ai_collection --skill spikerestormer-unified-event-reasoning. Copyright stays with the author (Complete terms in LICENSE.txt).

SpikeRestormer: Unified Event Reasoning for SNN Image Restoration

Overview

SpikeRestormer addresses the fundamental challenge of applying Spiking Neural Networks (SNNs) to static image restoration tasks. Traditional SNNs excel with dynamic event-based data but struggle with static images that lack explicit temporal events. SpikeRestormer introduces a unified event reasoning framework that generates internal spike-based degradation and restoration events, enabling energy-efficient all-in-one image restoration (AiOIR).

Core Components

1. Degradation-Event Perception Process

• Subtractive Degradation Event Attention (SDEA): Extracts spike-based degradation events from input images
• Converts static pixel information into temporal spike sequences representing degradation cues
• Handles diverse degradation types (noise, blur, compression artifacts) through spike event generation

2. Event-Reliability Inference Process

• Hierarchical Bayesian Skip Masking (HBSM): Infers reliability of degradation events across network hierarchy
• Uses Bayesian inference to determine which degradation events are trustworthy for restoration
• Provides skip connections with reliability-weighted masking for stable training

3. Restoration-Event Construction Process

• Additive Restoration Event Attention (AREA): Constructs restoration-oriented spike events
• Generates output spike sequences that represent the restored image content
• Combines degradation perception with reliability inference to produce final restoration events

Methodology Workflow

For Implementation

1. Input Processing: Convert static RGB/Grayscale image to initial spike representation
2. Degradation Event Extraction: Apply SDEA modules to generate degradation spike events
3. Reliability Assessment: Use HBSM to evaluate event trustworthiness at multiple scales
4. Restoration Construction: Employ AREA modules to build restoration spike events
5. Output Generation: Convert final spike events back to static image format

For Energy Efficiency Analysis

1. Calculate spike count reduction compared to traditional ANN approaches
2. Measure MAC/AC arithmetic energy savings in attention modules
3. Profile whole-model energy consumption during inference
4. Compare performance-per-watt metrics against baseline methods

Key Innovations

• Unified Event Reasoning: Formulates restoration as integrated process of degradation perception → reliability inference → restoration construction
• Internal Event Generation: Creates meaningful spike events from static inputs without external event cameras
• Energy-Efficient Design: Achieves significant energy reduction while maintaining competitive restoration quality
• All-in-One Capability: Handles multiple degradation types within single SNN architecture

Activation Keywords

• SpikeRestormer
• SNN image restoration
• Unified event reasoning
• Energy-efficient computer vision
• Static image SNN processing
• Degradation event attention
• Restoration event construction

Pitfalls and Considerations

Training Challenges

• Spike Generation Stability: Ensure consistent spike event generation across diverse degradation types
• Temporal Dynamics: Balance spike timing precision with computational efficiency
• Gradient Flow: Use surrogate gradient methods for backpropagation through spiking layers

Implementation Notes

• Hardware Compatibility: Optimize for neuromorphic hardware platforms (Loihi, SpiNNaker)
• Memory Requirements: Monitor spike buffer memory usage during high-resolution processing
• Quantization Effects: Consider impact of low-bit quantization on spike event quality

Validation Metrics

Performance Metrics

• PSNR (Peak Signal-to-Noise Ratio)
• SSIM (Structural Similarity Index)
• LPIPS (Learned Perceptual Image Patch Similarity)

Efficiency Metrics

• Total spike count
• Energy consumption (Joules per inference)
• MAC/AC operations reduction percentage
• Inference latency on target hardware

References

• Original Paper: arXiv:2608.02290
• Related Work: SeekBrain (arXiv:2607.29347), SMM Transformer (arXiv:2608.01622)
• SNN Fundamentals: Surrogate Gradient Learning, SpikingJelly Framework

When to Use This Skill

Use this skill when:

• Implementing energy-efficient image restoration systems
• Working with neuromorphic computing for computer vision tasks
• Researching SNN applications to static image processing
• Developing unified frameworks for multiple degradation handling
• Optimizing computer vision models for edge deployment with power constraints