Menu
Loading...
<

Guidelines

Time table Prizes Submission Rules Framework Contact

Timetable

  • Challenge Registration and Dry-run**: February 8 ~ April 30, 2021
  • Testing Phase and Result Submissions: April 30 (GMT+0 or UTC) ~ May 1, 2021 (GMT+0 or UTC)
  • Winner Validation and Confirmation Zip File Submission (see guidelines below): May 4, 2021 (GMT+0 or UTC)
  • Winner Announcement: May 20, 2021
** Dry-run phase: A set of dry-run (validation) data for each sub-track is provided for debugging purpose only. This set is provided just to validate your prediction format (so you can work with our final testing in the end). The Top-1 Accuracy on this set is not accurate due to small amount of data. Feedback are provided on this dry-run (validation) set only.

Prizes

  1. Action recognition from dark videos:
    1. Fully supervised action recognition in the dark
      • Winner (1st): one NVIDIA Titan V GPU, sponsored by NVIDIA
      • Runner-up (2nd): $800
      • Runner-up (3rd): $500
    2. Semi-supervised action recognition in the dark
      • Winner (1st): one NVIDIA Titan V GPU, sponsored by NVIDIA
      • Runner-up (2nd): $800
      • Runner-up (3rd): $500
For a total of about $8K net-worth awarded in prizes.

Submission Process

  • Submission format (before winner validation): A zipfile containing a single .csv file. A detailed description could be found at the following codalab pages under "Participate--Submit/View Results"
  • Codalab pages:
    1. Track 2.1: Codalab Link
    2. Track 2.2: Codalab Link
  • The "Team Name" in the Submission page should match the Team Name as registered in the Registration Form.
  • Test videos: downloadable April 30 (GMT+0 00:00) ~ May 1 (GMT+0 00:00).
  • Teams with Top-7 on the final testing Codalab leaderboard would go through a winner validation and confirmation process, see below for details.

If you have any questions about this challenge track please feel free to email cvpr2021.ug2challenge@gmail.com

Winner Validation and Confirmation Process

Teams with Top-7 on the final testing Codalab leaderboard would be required to submit their code for the winner validation and confirmation process. Please follow the instructions below carefully to ensure smooth confirmation process of your results:

  1. Download the training/validation/testing datasets. The testing dataset would be available for download on April 30 (GMT+0, 00:00).
  2. Train or finetune your model. For obtaining the final testing result, run your trained model on the testing set.
  3. Pack your code into one single .zip file. By default we would be using raw video as the input to your code. If your algorithm requires the use of individual video frame, include you frame extraction code as well in the zip file.
  4. Include a detailed instruction of the placement of the training, validation and testing data (i.e. where to place the data to enable training and testing).
  5. You should contain at most three running scripts, one for data pre-processing (named run_preproc.sh), one for training (named run_train.sh) and one for testing (inference) (named run_test.sh). We will ONLY run the three scripts during the validation process.
  6. Including the trained weight of your algorithm that could lead to the result shown in the leaderboard is highly recommended.
  7. Each team must only submit ONLY ONE zip file with ONLY ONE SET of algorithm corresponding to the submission result in the testing leaderboard. The zip file should contain all dependencies and code required to perform the algorithm's operation.

Running with NVIDIA Container Toolkit (nvidia-docker)

To ensure a smooth validation and confirmation process, we strongly recommend participants to run or verify their code with Docker and NVIDIA Container Toolkit (nvidia-docker). Here we provide detailed instructions to install and run nvidia-docker.

  1. Install NVIDIA GPU Driver using .run file. To ensure that you could run the required docker image, please install the GPU Driver with version >= 440.33.
  2. For the convenience of all participants, you may run the script provided here. Alternatively, follow the following steps.
  3. Install docker and nvidia-docker following the instructions provided by NVIDIA here. You may also choose to install older docker version following the instructions provided by Docker here.
  4. Our default docker image is pulled from NVIDIA NGC Catelog, the details of the docker is provided below. Follow the following two steps to pull and run the default docker image (also provided here or here):
    Using Pytorch:
    • docker pull nvcr.io/nvidia/pytorch:20.02-py3
    • docker run --gpus all -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/pytorch:20.02-py3 (Docker >= 19.03)
    • nvidia-docker run -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/pytorch:20.02-py3 (Docker <= 19.02)
    Using TensorFlow2:
    • docker pull nvcr.io/nvidia/tensorflow:20.03-tf2-py3
    • docker run --gpus all -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/tensorflow:20.03-tf2-py3 (Docker >= 19.03)
    • nvidia-docker run -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/tensorflow:20.03-tf2-py3 (Docker <= 19.02)
    Using TensorFlow1:
    • docker pull nvcr.io/nvidia/tensorflow:20.03-tf1-py3
    • docker run --gpus all -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/tensorflow:20.03-tf1-py3 (Docker >= 19.03)
    • nvidia-docker run -it --rm -v {local_dir}:{container_dir} nvcr.io/nvidia/tensorflow:20.03-tf1-py3 (Docker <= 19.02)
  5. Verify your algorithm using the running docker container. For more information on running docker containers, you may refer to the documentation here.

The use of Docker and nvidia-docker is NOT mandatory, but we would use the above docker images during the validation and confirmation process to ensure fair comparison between algorithms.

Requirements

Software
  • Docker-CE
  • NVIDIA Docker
  • NVIDIA GPU Driver >= 440.33
Docker

Our provided Docker will include the following software

For PyTorch:
  • Ubuntu 18.04
  • PyTorch 1.4.0a0+a5b4d78
  • NVIDIA CUDA 10.2.89
  • NVIDIA cuDNN 7.6.5
  • TensorBoard 2.1.0
For Tensorflow:
  • Ubuntu 18.04
  • TensorFlow 1.15.2 or TensorFlow 2.1.0
  • NVIDIA CUDA 10.2.89
  • NVIDIA cuDNN 7.6.5
Hardware

The proposed algorithms should be able to run in systems with:

  • Up to and including Titan Xp 12 GB
  • Up to and including 12 cores
  • Up to and including 32gb memory

If you have any questions about this challenge track please feel free to email cvpr2021.ug2challenge@gmail.com

Rules

Read carefully the following guidelines before submitting. Methods not complying with the guidelines will be disqualified.

  • We encourage participants to use the provided training and validation data for each task, as well as to make use of their own data or data from other sources for training. However the use any form of annotation or use of any of the provided benchmarks test sets for either supervised or unsupervised training is strictly forbidden.
  • Team name of submissions on Codalab must match the registration information. Any submission with a team name not registered will not be qualified for prizes. Only a single submission per team can be the winner of a single sub-track of this sub-challenge. Changes in algorithm parameters do not constitute a different method, all parameter tuning must be conducted using the dataset provided and any additional data the participants consider appropriate.

Eligibility

  • Foreign Nationals and International Developers: All Developers can participate with this exception: residents of, Iran, Cuba, North Korea, Crimea Region of Ukraine, Sudan or Syria or other countries prohibited on the U.S. State Department’s State Sponsors of Terrorism list. In addition, Developers are not eligible to participate if they are on the Specially Designated National list promulgated and amended, from time to time, by the United States Department of the Treasury. It is the responsibility of the Developer to ensure that they are allowed to export their technology solution to the United States for the Live Test. Additionally, it is the responsibility of participants to ensure that no US law export control restrictions would prevent them from participating when foreign nationals are involved. If there are US export control concerns, please contact the organizers and we will attempt to make reasonable accommodations if possible.

  • If you are entering as a representative of a company, educational institution or other legal entity, or on behalf of your employer, these rules are binding on you, individually, and/or the entity you represent or are an employee. If you are acting within the scope of your employment, as an employee, contractor, or agent of another party, you warrant that such party has full knowledge of your actions and has consented thereto, including your potential receipt of a prize. You further warrant that your actions do not violate your employer’s or entity’s policies and procedures.

  • The organizers reserve the right to verify eligibility and to adjudicate on any dispute at any time. If you provide any false information relating to the prize challenge concerning your identity, email address, ownership of right, or information required for entering the prize challenge, you may be immediately disqualified from the challenge.

  • Accounts. For the final testing phase, you may make submissions only under one, unique registration per team. Submission should be made such that the "Team Name" field in Codalab matches your team name in the registration. You will be disqualified if you make submissions for your final testing phase through more than one registration or if your team name cannot be found in the registration. Each TEAM may submit up to ten submissions per day per challenge. For the final winner validation/confirmation process, eligible teams should submit code containing only ONE algorithms per TEAM. Any submissions that does not adhere to this during the testing or winner validation/confirmation process may be subject to disqualification.

The organizers reserve the right to disqualify any participating team for any of the reasons mentioned above and if deemed necessary.

Warranty, indemnity and release

You warrant that your Submission is your own original work and, as such, you are the sole and exclusive owner and rights holder of the Submission, and you have the right to make the Submission and grant all required licenses. You agree not to make any Submission that: (i) infringes any third party proprietary rights, intellectual property rights, industrial property rights, personal or moral rights or any other rights, including without limitation, copyright, trademark, patent, trade secret, privacy, publicity or confidentiality obligations; or (ii) otherwise violates any applicable state or federal law.

To the maximum extent permitted by law, you indemnify and agree to keep indemnified challenge Entities at all times from and against any liability, claims, demands, losses, damages, costs and expenses resulting from any act, default or omission of the entrant and/or a breach of any warranty set forth herein. To the maximum extent permitted by law, you agree to defend, indemnify and hold harmless the challenge Entities from and against any and all claims, actions, suits or proceedings, as well as any and all losses, liabilities, damages, costs and expenses (including reasonable attorneys fees) arising out of or accruing from: (a) your Submission or other material uploaded or otherwise provided by you that infringes any copyright, trademark, trade secret, trade dress, patent or other intellectual property right of any person or entity, or defames any person or violates their rights of publicity or privacy; (b) any misrepresentation made by you in connection with the challenge; (c) any non-compliance by you with these Rules; (d) claims brought by persons or entities other than the parties to these Rules arising from or related to your involvement with the challenge; and (e) your acceptance, possession, misuse or use of any Prize, or your participation in the challenge and any challenge-related activity.

You hereby release organizers from any liability associated with: (a) any malfunction or other problem with the challenge Website; (b) any error in the collection, processing, or retention of any Submission; or (c) any typographical or other error in the printing, offering or announcement of any Prize or winners.

Competition Framework

The Participant has requested permission to use the dataset as compiled by Nanyang Technological University, Singapore, NVIDIA AI Technology Centre and University of Texas at Austin. In exchange for such permission, Participant hereby agrees to the following terms and conditions:

  • Nanyang Technological University, Singapore, NVIDIA AI Technology Centre and University of Texas at Austin make no representations or warranties regarding the Dataset, including but not limited to warranties of non-infringement or fitness for a particular purpose.

  • Pre-trained models are allowed in the competition.

  • Participants are not restricted to train their algorithms on the provided training set. Collecting and training on additional data is encouraged.

Contact

If you have any questions about this challenge track please feel free to email cvpr2021.ug2challenge@gmail.com

Footer