Oversampled binary image sensor: Difference between revisions

Revision as of 14:26, 18 March 2025

Latest revision as of 14:42, 8 June 2025

(One intermediate revision by one other user not shown)

Line 7:

The oversampled binary image sensor is reminiscent of photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local light intensity. At the start of the exposure period, all pixels are set to 0. A pixel is then set to 1 if the number of photons reaching it during the exposure is at least equal to a given threshold ''q''. One way to build such binary sensors is to modify standard memory chip technology, where each memory bit cell is designed to be sensitive to visible light.<ref name="RAMsensor">S. A. Ciarcia, A 64K-bit dynamic RAM chip is the visual sensor in this digital image camera, ''Byte Magazine'', pp.21-31, Sep. 1983.</ref> With current CMOS technology, the level of integration of such systems can exceed 10⁹~10¹⁰ (i.e., 1 giga to 10 giga) pixels per chip. In this case, the corresponding pixel sizes (around 50~nm <ref name="DRAM">Y. K. Park, S. H. Lee, J. W. Lee et al., Fully integrated 56nm DRAM technology for 1Gb DRAM, in ''IEEE Symposium on VLSI Technology'', Kyoto, Japan, Jun. 2007.</ref>) are far below the diffraction limit of light, and thus the image sensor is ''[[oversampling]]'' the optical resolution of the light field. Intuitively, one can exploit this spatial redundancy to compensate for the information loss due to one-bit quantizations, as is classic in oversampling [[delta-sigma converter]]s.<ref name="ODSC">J. C. Candy and G. C. Temes, Oversamling Delta-Sigma Data Converters-Theory, Design and Simulation. New York, NY: IEEE Press, 1992.</ref>

Building a binary sensor that emulates the photographic film process was first envisioned by [[Eric Fossum|Fossum]],<ref name="DFS">[[Eric Fossum|E. R. Fossum]], What to do with sub-diffraction-limit (SDL) pixels? - A proposal for a gigapixel digital film sensor (DFS), in ''IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors'', Nagano, Japan, Jun. 2005, pp.214-217.</ref> who coined the name ''digital film sensor'' (now referred to as a ''quanta image sensor''<ref>E.R. Fossum, J. Ma, S. Masoodian, L. Anzagira, and R. Zizza, ''The quanta image sensor: every photon counts'', MDPI Sensors, vol. 16, no. 8, 1260; August 2016. {{doi|10.3390/s16081260|doi-access=free}} (Special Issue on Photon-Counting Image Sensors)</ref>). The original motivation was mainly out of technical necessity. The [[miniaturization]] of camera systems calls for the continuous shrinking of pixel sizes. At a certain point, however, the limited [[full-well capacity]] (i.e., the maximum photon-electrons a pixel can hold) of small pixels becomes a bottleneck, yielding very low [[signal-to-noise ratio]]s (SNRs) and poor [[dynamic range]]s. In contrast, a binary sensor whose pixels need to detect only a few photon-electrons around a small threshold ''q'' has much less requirement for full-well capacities, allowing pixel sizes to shrink further.

==Imaging model==

Line 33:

==Reconstruction==

[[File:SPAD EPFL BINARY IMAGES.png|thumb|right|268px|Fig.4 Reconstructing an image from the binary measurements taken by a SPAD<ref name="SPADS">L. Carrara, C. Niclass, N. Scheidegger, H. Shea, and E. Charbon, A gamma, X-ray and high energy proton radiation-tolerant CMOS image sensor for space applications, in ''IEEE International Solid-State Circuits Conference'', Feb. 2009, pp.40-41.</ref> sensor, with a spatial resolution of 32×32 pixels. The final image (lower-right corner) is obtained by incorporating 4096 consecutive frames, 11 of which are shown in the figure.]]

One of the most important challenges with the use of an oversampled binary image sensor is the reconstruction of the light intensity <math>\lambda(x)</math> from the binary measurement <math>b_m</math>. [[Maximum likelihood|Maximum likelihood estimation]] can be used for solving this problem.<ref name="bitsfromphotons" /> Fig. 4 shows the results of reconstructing the light intensity from 4096 binary images taken by [[single photon avalanche diode]]s (SPADs) camera.<ref name="SPADS" /> A better reconstruction quality with fewer temporal measurements and faster, hardware friendly implementation, can be achieved by more sophisticated algorithms.<ref>{{Cite journal|title = Image reconstruction from dense binary pixels|arxiv = 1512.01774|journal = Signal Processing with Adaptive Sparse Structured Representations (SPARS 2015)|date = 2015-12-06|first = Or|last = Litany|first2 = Tal|last2 = Remez|first3 = Alex|last3 = Bronstein|bibcode = 2015arXiv151201774L}}</ref>