-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
125 lines (105 loc) · 5.93 KB
/
Copy pathindex.html
File metadata and controls
125 lines (105 loc) · 5.93 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
---
redirect_from: diffdiffdepth/
---
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Differentiable Diffusion for Dense Depth Estimation from Multi-view Images (CVPR 2021)</title>
<!-- Brown brand fonts (Minion 3 + Freight Sans), the lab's Adobe Fonts kit. -->
<link rel="preconnect" href="https://use.typekit.net" crossorigin>
<link rel="stylesheet" href="https://use.typekit.net/diw5pgv.css">
<!-- Shared BVC project-page styles + JS (bar, logo registry, theme). -->
<link rel="stylesheet" href="https://visual.cs.brown.edu/projects/shared/project.css">
<script defer src="https://visual.cs.brown.edu/projects/shared/project.js"></script>
<!-- Pre-paint theme: apply the stored/OS choice before first render (no
flash). Same localStorage 'theme' key + attribute as visual.cs.brown.edu. -->
<script>
(function () {
var t; try { t = localStorage.getItem('theme'); } catch (e) {}
var dark = t === 'dark' || (!t && matchMedia('(prefers-color-scheme: dark)').matches);
document.documentElement.setAttribute('data-bs-theme', dark ? 'dark' : 'light');
})();
</script>
</head>
<body>
<!-- The BVC bar (crest, section links, social icons, theme toggle) is
injected by the shared project.js. -->
<main class="project">
<h1 class="project-title">
Differentiable Diffusion for Dense Depth Estimation from Multi-view Images
</h1>
<p class="venue">Computer Vision and Pattern Recognition (CVPR) 2021</p>
<div class="authors">
<span class="author"><a href="https://cs.brown.edu/~nkhan6" target="_blank" rel="noopener">Numair Khan</a><sup>1</sup>,</span>
<span class="author"><a href="http://vclab.kaist.ac.kr/minhkim/" target="_blank" rel="noopener">Min H. Kim</a><sup>2</sup>,</span>
<span class="author"><a href="https://www.jamestompkin.com/" target="_blank" rel="noopener">James Tompkin</a><sup>1</sup></span>
</div>
<div class="affil-logos">
<span class="logo-item"><sup>1</sup><span data-logo="brown-cs"></span></span>
<span class="logo-item"><sup>2</sup><span data-logo="kaist"></span></span>
</div>
<nav class="resource-row">
<a href="./docs/khan2021_diffdiffdepth.pdf">Paper</a>
<a href="./docs/khan2021_diffdiffdepth_supp.pdf">Suppl</a>
<a href="https://arxiv.org/abs/2106.08917">arXiv</a>
<a href="https://github.com/brownvc/diffdiffdepth">Code</a>
<a href="#video">Video</a>
</nav>
<section id="abstract-section">
<h2 id="abstract">Abstract</h2>
<p>
We present a method to estimate dense depth by optimizing a sparse set of points such that their diffusion into a depth map minimizes a multi-view reprojection error from RGB supervision. We optimize point positions, depths, and weights with respect to the loss by differential splatting that models points as Gaussians with analytic transmittance. Further, we develop an efficient optimization routine that can simultaneously optimize the 50k+ points required for complex scene reconstruction. We validate our routine using ground truth data and show high reconstruction quality. Then, we apply this to light field and wider baseline images via self supervision, and show improvements in both average and outlier error for depth maps diffused from inaccurate sparse points. Finally, we compare qualitative and quantitative results to image processing and deep learning methods.
</p>
</section>
<section id="results-section">
<h2 id="results">Results</h2>
<div class="media-row">
<img src="./img/teaser_dino_rgb.png" alt="Reference RGB view of the dinosaur scene">
<img src="./img/teaser_dino_depth.gif" alt="Our diffused dense depth for the dinosaur scene">
</div>
<div class="media-row">
<img src="./img/teaser_lego_rgb.png" alt="Reference RGB view of the toy digger scene">
<img src="./img/teaser_lego_depth.gif" alt="Our diffused dense depth for the toy digger scene">
</div>
<p class="centerContent muted">Reference RGB view (left) and our diffused dense depth (right) for two scenes.</p>
</section>
<section id="video-section">
<h2 id="video">Video</h2>
<video class="center" style="display:block; width:100%;" controls preload="metadata">
<source src="./video/diffdiffdepth_cvpr2021.mp4" type="video/mp4">
</video>
<p class="centerContent"><a href="./video/diffdiffdepth_cvpr2021.mp4" download>Download video (MP4, 20 MB)</a></p>
</section>
<section id="citation-section">
<h2 id="citation">Citation</h2>
<pre class="citation"><code>@inproceedings{Khan_2021,
author = {Numair Khan and Min H. Kim and James Tompkin},
title = {Differentiable Diffusion for Dense Depth Estimation from Multi-view Images},
booktitle = {2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2021},
pages = {8908--8917},
doi = {10.1109/CVPR46437.2021.00880}
}</code></pre>
</section>
<section id="related-section">
<h2 id="related">Related Projects</h2>
<ul>
<li><a href="http://visual.cs.brown.edu/lightfielddepth/">4D Light Field Depth Estimation</a>—Efficient sparse estimation and view-consistent diffusion.</li>
<li><a href="https://github.com/brownvc/lightfieldsuperpixels/">4D Light Field Superpixels</a>—View-consistent and occlusion-aware estimation.</li>
</ul>
</section>
<section id="acknowledgements-section">
<h2 id="acknowledgements">Acknowledgements</h2>
<p>
We thank the reviewers for their detailed feedback. Numair Khan thanks an Andy van Dam PhD Fellowship, and Min H. Kim acknowledges the support of Korea NRF grant (2019R1A2C3007229).
</p>
</section>
<footer class="logos">
<span data-logo="brown-cs"></span>
<span data-logo="kaist"></span>
</footer>
</main>
</body>
</html>