Update readme

Author: Shamdan17

README.md

@@ -1,25 +1,43 @@
 # 🤔 ETA
 
-## Highlight
-We propose "Efficiency through Thinking Ahead" (ETA), an asynchronous dual-system that pre-processes information from past frames using a large model in tandem with processing the current information with a small model to enable real-time decisions with strong performance.
+## Highlight <a name="highlight"></a>
+We propose "**E**fficiency through **T**hinking **A**head" (ETA), an asynchronous dual-system that pre-processes information from past frames using a large model in tandem with processing the current information with a small model to enable real-time decisions with strong performance.
 <img width="800" alt="CarFormer overview" src="assets/overview.png">
 
-## Abstract
-How can we benefit from large models without sacrificing inference speed, a common dilemma in self-driving systems? A prevalent solution is a dual-system architecture, employing a small model for rapid, reactive decisions and a larger model for slower but more informative analyses. Existing dual-system designs often implement parallel architectures where inference is either directly conducted using the large model at each current frame or retrieved from previously stored inference results. However, these works still struggle to enable large models for a timely response to every online frame. Our key insight is to shift intensive computations of the current frame to previous time steps and perform a batch inference of multiple time steps to make large models respond promptly to each time step. To achieve the shifting, we introduce Efficiency through Thinking Ahead (ETA), an asynchronous system designed to: (1) propagate informative features from the past to the current frame using future predictions from the large model, (2) extract current frame features using a small model for real-time responsiveness, and (3) integrate these dual features via an action mask mechanism that emphasizes action-critical image regions. Evaluated on the Bench2Drive CARLA Leaderboard-v2 benchmark, ETA advances state-of-the-art performance by 8% with a driving score of 69.53 while maintaining a near-real-time inference speed at 50 ms.
+## News <a name="news"></a>
+- **`[2025/06/10]`** [ETA](https://arxiv.org/abs/2506.07725) paper and code release! 
 
 ## Results
 <img width="800" alt="CarFormer overview" src="assets/results.png">
 
-## Training and Inference
+
+## Table of Contents
+1. [Highlights](#highlight)
+2. [News](#news)
+3. [Results](#results)
+2. [Getting Started](#gettingstarted)
+   - [Training](docs/TRAIN_EVAL.md#trainingsetup)  
+   - [Evaluation](docs/TRAIN_EVAL.md#evalsetup)
+4. [TODO List](#todolist)
+6. [License and Citation](#licenseandcitation)
+7. [Other Resources](#otherresources)
+
+## Getting Started <a name="gettingstarted"></a>
 
 Please refer to [TRAIN_EVAL.md](docs/TRAIN_EVAL.md) for detailed instructions on how to train and evaluate the model.
 
+## TODO List
+- [x] ETA Training code
+- [x] ETA Evaluation
+  - [x] Inference Code
+  - [ ] Checkpoints
+
 ## Acknowledgements
 
-This codebase builds on open sourced code from [carla_garage]([email protected]:autonomousvision/carla_garage.git) among others. We thank the authors for their contributions. This project is funded by the European Union (ERC, ENSURE, 101116486) with additional compute support from Leonardo Booster (EuroHPC Joint Undertaking, EHPC-AI-2024A01-060). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. This study is also supported by National Natural Science Foundation of China (62206172) and Shanghai Committee of Science and Technology (23YF1462000).
+This codebase builds on open sourced code from [CARLA Garage]([email protected]:autonomousvision/carla_garage.git) and [Bench2DriveZoo](https://github.com/Thinklab-SJTU/Bench2DriveZoo/) among others. We thank the authors for their contributions. This project is funded by the European Union (ERC, ENSURE, 101116486) with additional compute support from Leonardo Booster (EuroHPC Joint Undertaking, EHPC-AI-2024A01-060). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. This study is also supported by National Natural Science Foundation of China (62206172) and Shanghai Committee of Science and Technology (23YF1462000).
 
-## Citation
-If you find our project useful for your research, please consider citing our paper with the following BibTeX:
+## License and Citation <a name="licenseandcitation"></a>
+This project is released under the [MIT License](LICENSE). If you find our project useful for your research, please consider citing our paper with the following BibTeX:
 
 
 ```bibtex

assets/overview.png

@@ -17,7 +17,6 @@ def main(cfg):
 
     args = cfg
 
-    # NCCL_DEBUG=WARN HF_ENDPOINT=https://hf-mirror.com OMP_NUM_THREADS=6 python  train_hydra_ds_lightning.py deepspeed=zero2 hyperparams.batch_size=16 hyperparams.num_epochs=40 user=shadihamdan wandb_tag=finalrun_v6dlc_basedata_llava1pt6 training/bev/[email protected]_backbone=llava1pt6 dataset.dataset_path_rel=B2D-base gpus=8 hyperparams.gradient_accumulation_steps=2 training.parallel_dataset_init=False 'ckpt_path="/cpfs01/user/shadihamdan/research/Carformer/checkpoints/TRAINING/bs=16_gradient_accumulation_steps=2_eps=40_training_bev_rgb_backbonergb_backbone=llava1pt6_wnb=finalrun_v6dlc_basedata_llava1pt6_data=B2D-base_bev=rgb_front/2024-08-20_18-53-38/last.ckpt"'
     if args.ckpt_path is not None:
         print("Loading model from checkpoint")
         cfg.save_dir = os.path.dirname(args.ckpt_path)

carformer/train.py

@@ -1,4 +1,4 @@
-## Training Setup
+## Training Setup <a name="trainingsetup"></a>
 ### Bench2Drive data
 Download the [Bench2Drive](https://github.com/Thinklab-SJTU/Bench2Drive) dataset and unzip all the directories. 
 
@@ -30,7 +30,7 @@ make ETA_base_model_s42
 make ETA_async_model_s42
 ```
 
-## Evaluation Setup
+## Evaluation Setup <a name="evalsetup"></a>
 
 ### Bench2Drive