TypeError: file must have 'read', 'readinto' and 'readline' attributes

[goinouego]

Hello.
This is my first post, so please excuse me if there are any mistakes.

The commands used are as follows.

cd reincarnating_rl
python -um reincarnating_rl.train \
  --agent qdagger_rainbow \
  --gin_files reincarnating_rl/configs/qdagger_rainbow.gin
  --base_dir /tmp/qdagger_rainbow \
  --teacher_checkpoint_dir $TEACHER_CKPT_DIR/Breakout/1 \
  --teacher_checkpoint_number 399
  --run_number=1 \
  --atari_roms_path=/tmp/atari_roms \
  --alsologtostderr

・'atari_roms_path' was not listed, so I added it myself.

When I run the code, the following error occurs.

 2023-10-04 13:09:32.049601: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-10-04 13:09:33.558113: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/home/mlab/tensei2/lib/python3.8/site-packages/jax/_src/api_util.py:229: SyntaxWarning: Jitted function has invalid argnames {'distill_type', 'distill_loss_coefficient', 'distill_best_action_only', 'distill_temperature'} in static_argnames. Function does not take these args.This warning will be replaced by an error after 2022-08-20 at the earliest.
  warnings.warn(f"Jitted function has invalid argnames {invalid_argnames} "
2023-10-04 13:09:38.179840: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-10-04 13:09:38.183743: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
I1004 13:09:38.941833 140563993347904 train.py:125] Setting random seed: 1
W1004 13:09:41.145547 140563993347904 run_experiment.py:267] DEPRECATION WARNING: Logger is being deprecated. Please switch to CollectorDispatcher!
/home/mlab/tensei2/lib/python3.8/site-packages/gym/envs/registration.py:593: UserWarning: WARN: The environment PongNoFrameskip-v0 is out of date. You should consider upgrading to version `v4`.
  logger.warn(
A.L.E: Arcade Learning Environment (version 0.7.5+db37282)
[Powered by Stella]
/home/mlab/tensei2/lib/python3.8/site-packages/gym/core.py:317: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
/home/mlab/tensei2/lib/python3.8/site-packages/gym/wrappers/step_api_compatibility.py:39: DeprecationWarning: WARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
I1004 13:09:43.154085 140563993347904 reincarnation_rainbow_agent.py:43] Creating ReincarnationRainbowAgent agent with the following parameters:
I1004 13:09:43.154490 140563993347904 reincarnation_rainbow_agent.py:45]         num_updates_per_persistent_step: 1
I1004 13:09:43.154817 140563993347904 reincarnation_rainbow_agent.py:47]         data_augmentation: False
I1004 13:09:43.155117 140563993347904 reincarnation_rainbow_agent.py:48]         network: <class 'dopamine.jax.networks.FullRainbowNetwork'>
I1004 13:09:43.752246 140563993347904 full_rainbow_agent.py:224] Creating QDaggerRainbowAgent agent with the following parameters:
I1004 13:09:43.753766 140563993347904 full_rainbow_agent.py:226]         double_dqn: False
I1004 13:09:43.754319 140563993347904 full_rainbow_agent.py:227]         noisy_networks: False
I1004 13:09:43.755004 140563993347904 full_rainbow_agent.py:228]         dueling_dqn: False
I1004 13:09:43.755769 140563993347904 full_rainbow_agent.py:229]         distributional: True
I1004 13:09:43.756666 140563993347904 full_rainbow_agent.py:230]         mse_loss: 0
I1004 13:09:43.757438 140563993347904 full_rainbow_agent.py:231]         num_atoms: 51
I1004 13:09:43.758368 140563993347904 full_rainbow_agent.py:232]         replay_scheme: prioritized
I1004 13:09:43.759295 140563993347904 full_rainbow_agent.py:233]         num_updates_per_train_step: 1
I1004 13:09:43.814832 140563993347904 xla_bridge.py:603] Unable to initialize backend 'cuda': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I1004 13:09:43.815029 140563993347904 xla_bridge.py:603] Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I1004 13:09:43.816573 140563993347904 xla_bridge.py:603] Unable to initialize backend 'tpu': INVALID_ARGUMENT: TpuPlatform is not available.
W1004 13:09:43.816766 140563993347904 xla_bridge.py:617] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
I1004 13:09:43.989467 140563993347904 dqn_agent.py:278] Creating QDaggerRainbowAgent agent with the following parameters:
I1004 13:09:43.989721 140563993347904 dqn_agent.py:280]          gamma: 0.990000
I1004 13:09:43.989921 140563993347904 dqn_agent.py:281]          update_horizon: 3.000000
I1004 13:09:43.990098 140563993347904 dqn_agent.py:282]          min_replay_history: 20000
I1004 13:09:43.990217 140563993347904 dqn_agent.py:283]          update_period: 4
I1004 13:09:43.990343 140563993347904 dqn_agent.py:284]          target_update_period: 8000
I1004 13:09:43.990493 140563993347904 dqn_agent.py:285]          epsilon_train: 0.010000
I1004 13:09:43.990720 140563993347904 dqn_agent.py:286]          epsilon_eval: 0.001000
I1004 13:09:43.990818 140563993347904 dqn_agent.py:287]          epsilon_decay_period: 50000
I1004 13:09:43.990914 140563993347904 dqn_agent.py:288]          optimizer: adam
I1004 13:09:43.991011 140563993347904 dqn_agent.py:289]          seed: 1
I1004 13:09:43.991108 140563993347904 dqn_agent.py:290]          loss_type: mse
I1004 13:09:43.991207 140563993347904 dqn_agent.py:291]          preprocess_fn: <function preprocess_inputs_with_augmentation at 0x7fd6d206a940>
I1004 13:09:43.991539 140563993347904 dqn_agent.py:292]          summary_writing_frequency: 25000
I1004 13:09:43.991641 140563993347904 dqn_agent.py:293]          allow_partial_reload: False
I1004 13:09:45.225142 140563993347904 circular_replay_buffer.py:157] Creating a OutOfGraphPrioritizedReplayBuffer replay memory with the following parameters:
I1004 13:09:45.225919 140563993347904 circular_replay_buffer.py:160]     observation_shape: (84, 84)
I1004 13:09:45.226537 140563993347904 circular_replay_buffer.py:161]     observation_dtype: <class 'jax.numpy.uint8'>
I1004 13:09:45.227458 140563993347904 circular_replay_buffer.py:162]     terminal_dtype: <class 'numpy.uint8'>
I1004 13:09:45.228268 140563993347904 circular_replay_buffer.py:163]     stack_size: 4
I1004 13:09:45.229049 140563993347904 circular_replay_buffer.py:164]     replay_capacity: 1000000
I1004 13:09:45.229805 140563993347904 circular_replay_buffer.py:165]     batch_size: 32
I1004 13:09:45.230160 140563993347904 circular_replay_buffer.py:166]     update_horizon: 3
I1004 13:09:45.230264 140563993347904 circular_replay_buffer.py:167]     gamma: 0.990000
I1004 13:09:45.230487 140563993347904 circular_replay_buffer.py:168]     checkpoint_duration: 4
I1004 13:09:45.230724 140563993347904 circular_replay_buffer.py:169]     keep_every: None
I1004 13:10:13.732142 140563993347904 loss_helpers.py:83] Creating Adam optimizer with settings lr=0.000063, beta1=0.900000, beta2=0.999000, eps=0.000150
I1004 13:10:14.991514 140563993347904 atari_100k_rainbow_agent.py:159]   data_augmentation: False
I1004 13:10:14.992304 140563993347904 atari_100k_rainbow_agent.py:162]   data_augmentation: False
I1004 13:10:14.993070 140563993347904 qdagger_rainbow_agent.py:158] Creating QDaggerRainbowAgent agent with the following parameters:
I1004 13:10:14.993680 140563993347904 qdagger_rainbow_agent.py:160]      distill_decay_period: 2500000
I1004 13:10:14.994260 140563993347904 qdagger_rainbow_agent.py:161]      distill_loss_coefficient: 1.0000
I1004 13:10:14.994835 140563993347904 qdagger_rainbow_agent.py:162]      distill_best_action_only: 0.0000
I1004 13:10:14.995398 140563993347904 qdagger_rainbow_agent.py:163]      distill_temperature: 0.1000
I1004 13:10:14.995963 140563993347904 qdagger_rainbow_agent.py:164]      method_type: 1
Traceback (most recent call last):
  File "/home/mlab/.pyenv/versions/3.8.0/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/mlab/.pyenv/versions/3.8.0/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/mlab/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/mlab/tensei2/reincarnating_rl-master/train.py", line 173, in <module>
    app.run(main)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/mlab/tensei2/reincarnating_rl-master/train.py", line 161, in main
    runner = runner_fn(
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/mlab/tensei2/reincarnating_rl-master/reincarnating_rl/run_experiment.py", line 301, in __init__
    super().__init__(
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/mlab/tensei2/reincarnating_rl-master/reincarnating_rl/run_experiment.py", line 76, in __init__
    super().__init__(
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/mlab/tensei2/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/dopamine/discrete_domains/run_experiment.py", line 235, in __init__
    self._initialize_checkpointer_and_maybe_resume(checkpoint_file_prefix)
  File "/home/mlab/tensei2/reincarnating_rl-master/reincarnating_rl/run_experiment.py", line 116, in _initialize_checkpointer_and_maybe_resume
    experiment_data = self._checkpointer.load_checkpoint(checkpoint_version)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/dopamine/discrete_domains/checkpointer.py", line 214, in load_checkpoint
    return self._load_data_from_file(checkpoint_file)
  File "/home/mlab/tensei2/lib/python3.8/site-packages/dopamine/discrete_domains/checkpointer.py", line 193, in _load_data_from_file
    return pickle.load(fin)
TypeError: file must have 'read', 'readinto' and 'readline' attributes
  In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
  In call to configurable 'RunnerWithTeacher' (<class 'reincarnating_rl.run_experiment.RunnerWithTeacher'>)
  In call to configurable 'ReincarnationRunner' (<class 'reincarnating_rl.run_experiment.ReincarnationRunner'>)

I get the error: Typeerror: file must have 'read', 'readinto' and 'readline' attributes.

My system info:
・Ubuntu 18.04
・Python 3.8.0
・I installed dependencies from requirements.txt.
・All checkpoint files have been downloaded and placed in a directory named DQN_400 at the same level as the README file.

I would appreciate it if you could reply.

[sdpkjc]

I've also come across some quirky bugs in the latest version 😬, which I suspect got slipped in during code refactoring.

Going through the init commit from git, it appeared a tad more accurate 🕵️.

May I suggest you to cast your eyes on distillation_dqn_agent.py and persistent_dqn_agent.py? They might just lend you a helping hand 🤝.

By the way, CleanRL's QDagger(code & docs) is quite straightforward to reproduce🚀. I highly recommend giving it a try!

[agarwl]

Yes, this might have happened during refactoring and I'd also suggest looking at CleanRL's code for a runnable version. The code here can be useful as a reference for implementation.