Please see more competition information in https://github.com/360SR/360SR-Challenge.
Abstract
The 360° or omnidirectional images/videos can provide an immersive and interactive experience and have received much research attention with the popularity of AR/VR applications. Unlike planar images/videos that have a narrow field of view (FoV), 360° images/videos can represent the whole scene in all directions. However, 360° images/videos suffer from the lower angular resolution problem since they are captured by fisheye lens with the same sensor size for capturing planar images. Although the whole 360° images/videos are of high resolution, the details are not satisfying. In many application scenarios, increasing the resolution of 360° images/videos is highly demanded to achieve higher perceptual quality and boost the performance of downstream tasks.
Recently, considerable success has been achieved in the image and video super-resolution (SR) task with the development of deep learning-based methods. Although 360° images/videos are often transformed into 2D planar representations by preserving omnidirectional information in practice, like equirectangular projection (ERP) and cube map projection (CMP), existing super-resolution methods still cannot be directly applied to 360° images/videos due to the distortions introduced by the projections. As for videos, the temporal relationships in a 360° video should be further considered since it is different from that in an ordinary 2D video. Therefore, how to effectively super-resolve 360° image/video by considering these characteristics remains challenging.
To rectify the lack of high-quality datasets in the community of omnidirectional image/video super-resolution, we construct new 360° datasets for image (Flickr360) and video (ODV360), respectively.
Flickr360
Flickr360 contains about 3150 ERP images with a resolution larger than 5k. Specifically, 3100 images are collected from Flickr, and the other 50 images are captured by Insta360° cameras. The images from Flickr are under either Creative Commons BY 2.0, Creative Commons BY-NC 2.0, Public Domain Mark 1.0, Public Domain CC0 1.0, or U.S. Government Works license. All of these licenses allow free use, redistribution, and adaptation for non-commercial purposes. The image contents vary both indoors and outdoors. We first downsample the original images into 2k resolution (2048 x 1024), serving as HR images. These HR images are further downsampled into LR images. The data partition is shown in the following table.
| Training | Validation | Testing | |
|---|---|---|---|
| Source | Flickr 360 | Flickr 360 | Flickr 360+capturing |
| Number | 3000 | 50 | 50+50 |
| Storage | 8.1G (HR) + 553M (LR) | 137M (HR) + 9.3M (LR) | 271M (HR) + 20M (LR) |
Download
● 腾讯微云
Samples
See here for samples.
ODV360
ODV360 including two parts:
● 90 videos collected from YouTube and public 360° video dataset
These videos are carefully selected and have high quality to be used for restoration. All videos have the license of Creative Commons Attribution license (reuse allowed), and our dataset is used only for academic and research proposes
● 160 videos collected by ourselves with Insta360 cameras
The cameras we use include Insta 360 X2 and Insta 360 ONE RS. They can capture high-resolution (5.7K) omnidirectional videos.
These collected omnidirectional videos cover a large range of diversity, and the video contents vary indoors and outdoors. To facilitate the use of these videos for research, we downsample the original videos into 2K resolution (2160x1080) by OpenCV. The number of frames per video is fixed at about 100. We randomly divide these videos into training, validation, and testing sets, as shown in the following table.
| Training | Validation | Testing | All | |
|---|---|---|---|---|
| Number | 210 | 20 | 20 | 250 |
| Storage | GT(59G)+LR(4.9G) | GT(5.3G)+LR(446M) | GT(5.7G)+LR(485M) | 75.8G |
Download
● 腾讯微云
Samples
See here for samples.
Contact
If you have any question, please open an issue in https://github.com/360SR/360SR-Challenge.