News!

    l April 6 2023：The test phase begins. The test dataset for Tracks 1 & 2 has been released！

     l April 7 2023：The test dataset for Tracks 3 & 4 has been released！

     l Apr. 20, 2023：The notification of winners is now released.

       The valid submission only if the participants have submitted their paper and results could be considered as the winner. We will invite the winners of the competition (top three) to share their winning solutions at ICME and award certificates to the top three.

Background

Degraded multimedia data, especially real rain, poses serious challenges to scene understanding. Although existing computer vision technologies have achieved impressive results under unreal synthetic conditions, especially for those with simple degradation factors, they often lose their ability on real-world rain scene images, in which the background content has suffered various complex degradations. In outdoor applications such as automatic driving and monitoring systems, in particular, the degradation factors on rainy days involve rain intensity, occlusion, blur, droplets, reflection, wiper, and their blending effects, commonly confusing scene contents and causing misunderstandings. Consequently, these adverse effects often degrade performances of various vision tasks, such as scene understanding, object detection, and identification. There is an urgent need to deliver efforts to seek the optimized solution to meeting the actual needs of the people's livelihood.

Challenge

   (a)Droplet                     (b)Reflection                 (c)Before wiper            (d)After wiper

Shading Effect of Droplet (a)：Droplet, the most common phenomenon on rainy days, will prevent light from passing straight through the object, and when it falls on the windshield, the content distortion caused by the reflectance and refraction becomes more pronounced. In Fig. (a), areas covered with rain droplet has significant misidentification, the green "vegetation'' is identified as the grey "building'', while the blue "sky'' is misidentified as the grey "building''.

Road Reflection (b)：The rain droplets accumulated on the road commonly form a mirror reflection effect, confusing the segmentation model with the fake and indistinguishable objective boundary. As shown in Fig. (b), the shadow of cars is incorrectly identified.

Blurring by Windshield Wiper (c)：The wiper eliminates the visual occlusion caused by the rain accumulation. However, for the real rainy days, the wiper operation could evidently promote the visual clarity of human observed views, but the unexpected change of scene contents is unfriendly to the segmentation model for the real-time application. As is shown in the yellow box in Fig. (c), the result is not consistent with the human's view.

Timeline

Grand Challenge papers have the same camera-ready deadline as regular papers, subject to the final release time of ICME. Please refer to the important dates for details.

Rainy WCity Datasets

Introduce

The Rainy WCity dataset consists of 300 rainy images with handcrafted annotations. We have observed that there are three typical degradation factors in these images that affect the quality of the image: raindrop, wiper, and reflection. Here we split the Rainy WCity with 600 images for training and 100 for testing, and each segment divided into three main categories according to the three degradation factors. The size of each image is 1,920×1,080. And there is no pixel-by-pixel paired ground-truth reference images, since our dataset is taken in the real rain scene. Note that the training data is composed of 400 images without corresponding annotations and 200 images with annotations.

Rainy WCity covers various rain patterns and their bring-in negative visual effects, covering raindrop, wiper, reflection, refraction, shadow, windshield-blurring, etc. Our dataset distinguishes others by featuring the following elements: 1) Diverse Rain Patterns: including light, moderate, and heavy rain scenarios; 2) Diverse degradation factors: besides the typical rain occlusion, our dataset considers numerous adverse effects of degradation, commonly occurred on rainy days but overlooked in other real rainy datasets, e.g., the raindrop interference, windshield-blurring, and road reflection.

In modern digital cameras, the output image we get from the camera is not the raw image, but rather the image obtained by passing the raw sensor data through the image signal processing (ISP) pipeline. In practice, if the ISP pipeline does not perform a denoising step, the processed sensor noise would deteriorate the output image by introducing non-Gaussian noise. Therefore, in Tracks 3 & 4, we processed the image in order to simulate the raw degradation scene.Specifically, we set up a mixture of random probability noise, adopting JPEG compression with different quality factors, and generating processed camera sensor noise via reverse-forward camera image signal processing (ISP) pipeline model and RAW image noise model. we perform randomly shuffled degradations to synthesize raw degradation images.This challenge aims at addressing the visibility problem of vision tasks under adverse weather scenarios (e.g.,fog, rain, low-illumination) for sharing ideas and discussions on current trends, issues, and future directions. The topics include but are not limited to:

Topic

This challenge aims at addressing the visibility problem of vision tasks under adverse weather scenarios (e.g.,fog, rain, low-illumination) for sharing ideas and discussions on current trends, issues, and future directions. The topics include but are not limited to:

l Semantic Segmentation under Real Rain Scene started

l Object Detection under Real Rain Scene started

l Semantic Segmentation under Simulated Raw Degradation Scene started

l Object Detection under Simulated Raw Degradation Scene started

Download

Baidu Netdisk：Rainy WCity

Google Drive：Rainy WCity

Ask for Permission

Track 1: Semantic Segmentation under Real Rain Scene

Track 2: Object Detection under Real Rain Scene

Track 3: Semantic Segmentation under Simulated Raw Degradation Scene

Track 4: Object Detection under Simulated Raw Degradation Scene

If you would like to use our dataset, please download the release file. The download of Rainy WCity DATABASE RELEASE AGREEMENT is here：

      Rainy Wcity DATAEBASE RELEASE.docx

Evaluation Protocol

Semantic Segmentation：mIoU

Object Detection：ImAP

Real Rain Scene：NIQE、SSEQ、PIQE、Perceptual Index、BIoU

Simulated Real Degradation Scene：PSNR、SSIM

l In the semantic segmentation field, metrics such as Mean Intersection over Union (mIoU) are commonly used to evalute segmentation effects.

l In the object detection field, conventional metrics like Mean Average Precision (mAP). is used to measure detection effectiveness.

l (Additional) In the real-world, since our real rain scene belongs to the field of image enhancement, other non-referenced subjective evaluation metrics are also available: e.g. Spatial-Spectral Entropy-based Quality (SSEQ), Natural Image Quality Evaluator (NIQE), Perception-based Image Quality Evaluator (PIQE) 、Perception Index (PI) and Boundary IoU(BIoU).

l (Additional) To measure the fidelity, we use the standard Peak Signal to Noise Ratio (PSNR) and the Structural Similarity (SSIM) index.

Paper Submission

l Submission deadline

l Length: Papers must be no longer than 6 pages, including all text, figures, and references.

l Format: Grand Challenge papers have the same format as regular papers. See the example paper under the General Information section below. However, their review is single blind.

l Submission: Submit the written component via CMT under the appropriate Grand Challenge track. Submit the data component, if any, directly to the Grand Challenge organizers as specified on the appropriate Grand Challenge site.

l Review: Submissions of both written and data components will be reviewed directly by the Grand Challenge organizers. Accepted submissions (written component only) will be included in the USB Proceedings and the authors will be given the opportunity to present their work at ICME. “Winning” submissions will be announced by the Grand Challenge organizers at the conference.

l Submissions may be accompanied by up to 20 MB of supplemental material following the same guidelines as regular and special session papers.

l Presentation guarantee: As with accepted Regular and Special Session papers, accepted Grand Challenge papers must be registered by the author deadline and presented at the conference; otherwise they will not be included in IEEE Xplore.

l A Grand Challenge paper is covered by a full-conference registration only.

l Note：Submitted GC papers can properly have a missing results section, which should be completed after GC organizers have completed their evaluations.

Organizers

Results

Tracks 1 & 3: Semantic Segmentation under Real Rain Scene

Note：we will select the top three winners in Track 1 and 3 respectively.

Track 2: Object Detection under Real Rain Scene

Note：we will select the top winners in Track 2.

Communication & QA

zhongx@whut.edu.cn