diff --git "a/sf_log.txt" "b/sf_log.txt"
new file mode 100644--- /dev/null
+++ "b/sf_log.txt"
@@ -0,0 +1,989 @@
+[2024-07-24 17:47:35,504][03928] Saving configuration to /content/train_dir/default_experiment/config.json...
+[2024-07-24 17:47:35,509][03928] Rollout worker 0 uses device cpu
+[2024-07-24 17:47:35,510][03928] Rollout worker 1 uses device cpu
+[2024-07-24 17:47:35,512][03928] Rollout worker 2 uses device cpu
+[2024-07-24 17:47:35,514][03928] Rollout worker 3 uses device cpu
+[2024-07-24 17:47:35,516][03928] Rollout worker 4 uses device cpu
+[2024-07-24 17:47:35,517][03928] Rollout worker 5 uses device cpu
+[2024-07-24 17:47:35,518][03928] Rollout worker 6 uses device cpu
+[2024-07-24 17:47:35,519][03928] Rollout worker 7 uses device cpu
+[2024-07-24 17:47:35,676][03928] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2024-07-24 17:47:35,678][03928] InferenceWorker_p0-w0: min num requests: 2
+[2024-07-24 17:47:35,720][03928] Starting all processes...
+[2024-07-24 17:47:35,721][03928] Starting process learner_proc0
+[2024-07-24 17:47:36,990][03928] Starting all processes...
+[2024-07-24 17:47:37,002][03928] Starting process inference_proc0-0
+[2024-07-24 17:47:37,003][03928] Starting process rollout_proc0
+[2024-07-24 17:47:37,005][03928] Starting process rollout_proc1
+[2024-07-24 17:47:37,007][03928] Starting process rollout_proc2
+[2024-07-24 17:47:37,007][03928] Starting process rollout_proc3
+[2024-07-24 17:47:37,008][03928] Starting process rollout_proc4
+[2024-07-24 17:47:37,009][03928] Starting process rollout_proc5
+[2024-07-24 17:47:37,009][03928] Starting process rollout_proc6
+[2024-07-24 17:47:37,009][03928] Starting process rollout_proc7
+[2024-07-24 17:47:52,502][05149] Worker 0 uses CPU cores [0]
+[2024-07-24 17:47:52,703][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2024-07-24 17:47:52,706][05135] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
+[2024-07-24 17:47:52,719][05153] Worker 4 uses CPU cores [0]
+[2024-07-24 17:47:52,737][05156] Worker 5 uses CPU cores [1]
+[2024-07-24 17:47:52,777][05135] Num visible devices: 1
+[2024-07-24 17:47:52,780][05152] Worker 3 uses CPU cores [1]
+[2024-07-24 17:47:52,798][05151] Worker 2 uses CPU cores [0]
+[2024-07-24 17:47:52,808][05135] Starting seed is not provided
+[2024-07-24 17:47:52,808][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2024-07-24 17:47:52,808][05135] Initializing actor-critic model on device cuda:0
+[2024-07-24 17:47:52,809][05135] RunningMeanStd input shape: (3, 72, 128)
+[2024-07-24 17:47:52,812][05135] RunningMeanStd input shape: (1,)
+[2024-07-24 17:47:52,819][05150] Worker 1 uses CPU cores [1]
+[2024-07-24 17:47:52,823][05155] Worker 7 uses CPU cores [1]
+[2024-07-24 17:47:52,829][05148] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2024-07-24 17:47:52,829][05148] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
+[2024-07-24 17:47:52,852][05148] Num visible devices: 1
+[2024-07-24 17:47:52,858][05135] ConvEncoder: input_channels=3
+[2024-07-24 17:47:52,879][05154] Worker 6 uses CPU cores [0]
+[2024-07-24 17:47:53,079][05135] Conv encoder output size: 512
+[2024-07-24 17:47:53,079][05135] Policy head output size: 512
+[2024-07-24 17:47:53,131][05135] Created Actor Critic model with architecture:
+[2024-07-24 17:47:53,132][05135] ActorCriticSharedWeights(
+  (obs_normalizer): ObservationNormalizer(
+    (running_mean_std): RunningMeanStdDictInPlace(
+      (running_mean_std): ModuleDict(
+        (obs): RunningMeanStdInPlace()
+      )
+    )
+  )
+  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
+  (encoder): VizdoomEncoder(
+    (basic_encoder): ConvEncoder(
+      (enc): RecursiveScriptModule(
+        original_name=ConvEncoderImpl
+        (conv_head): RecursiveScriptModule(
+          original_name=Sequential
+          (0): RecursiveScriptModule(original_name=Conv2d)
+          (1): RecursiveScriptModule(original_name=ELU)
+          (2): RecursiveScriptModule(original_name=Conv2d)
+          (3): RecursiveScriptModule(original_name=ELU)
+          (4): RecursiveScriptModule(original_name=Conv2d)
+          (5): RecursiveScriptModule(original_name=ELU)
+        )
+        (mlp_layers): RecursiveScriptModule(
+          original_name=Sequential
+          (0): RecursiveScriptModule(original_name=Linear)
+          (1): RecursiveScriptModule(original_name=ELU)
+        )
+      )
+    )
+  )
+  (core): ModelCoreRNN(
+    (core): GRU(512, 512)
+  )
+  (decoder): MlpDecoder(
+    (mlp): Identity()
+  )
+  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
+  (action_parameterization): ActionParameterizationDefault(
+    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
+  )
+)
+[2024-07-24 17:47:53,393][05135] Using optimizer <class 'torch.optim.adam.Adam'>
+[2024-07-24 17:47:54,125][05135] No checkpoints found
+[2024-07-24 17:47:54,125][05135] Did not load from checkpoint, starting from scratch!
+[2024-07-24 17:47:54,125][05135] Initialized policy 0 weights for model version 0
+[2024-07-24 17:47:54,128][05135] LearnerWorker_p0 finished initialization!
+[2024-07-24 17:47:54,129][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2024-07-24 17:47:54,268][05148] RunningMeanStd input shape: (3, 72, 128)
+[2024-07-24 17:47:54,269][05148] RunningMeanStd input shape: (1,)
+[2024-07-24 17:47:54,282][05148] ConvEncoder: input_channels=3
+[2024-07-24 17:47:54,385][05148] Conv encoder output size: 512
+[2024-07-24 17:47:54,385][05148] Policy head output size: 512
+[2024-07-24 17:47:54,439][03928] Inference worker 0-0 is ready!
+[2024-07-24 17:47:54,441][03928] All inference workers are ready! Signal rollout workers to start!
+[2024-07-24 17:47:54,722][05155] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,728][05152] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,762][05149] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,764][05153] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,772][05151] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,774][05156] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,767][05154] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:54,861][05150] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 17:47:55,668][03928] Heartbeat connected on Batcher_0
+[2024-07-24 17:47:55,671][03928] Heartbeat connected on LearnerWorker_p0
+[2024-07-24 17:47:55,714][03928] Heartbeat connected on InferenceWorker_p0-w0
+[2024-07-24 17:47:56,388][03928] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2024-07-24 17:47:56,608][05155] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:56,608][05152] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:56,895][05154] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:56,907][05153] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:56,911][05149] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:56,917][05151] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:59,290][05155] Decorrelating experience for 32 frames...
+[2024-07-24 17:47:59,814][05156] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:59,825][05154] Decorrelating experience for 32 frames...
+[2024-07-24 17:47:59,819][05150] Decorrelating experience for 0 frames...
+[2024-07-24 17:47:59,856][05152] Decorrelating experience for 32 frames...
+[2024-07-24 17:47:59,867][05149] Decorrelating experience for 32 frames...
+[2024-07-24 17:47:59,905][05151] Decorrelating experience for 32 frames...
+[2024-07-24 17:48:00,209][05153] Decorrelating experience for 32 frames...
+[2024-07-24 17:48:01,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2024-07-24 17:48:02,102][05151] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:02,329][05156] Decorrelating experience for 32 frames...
+[2024-07-24 17:48:02,354][05150] Decorrelating experience for 32 frames...
+[2024-07-24 17:48:02,506][05155] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:03,928][05155] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:04,097][05150] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:04,103][03928] Heartbeat connected on RolloutWorker_w7
+[2024-07-24 17:48:04,570][05154] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:04,595][05151] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:05,011][05149] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:05,180][03928] Heartbeat connected on RolloutWorker_w2
+[2024-07-24 17:48:06,098][05150] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:06,102][05156] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:06,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2024-07-24 17:48:06,392][03928] Avg episode reward: [(0, '1.280')]
+[2024-07-24 17:48:06,648][03928] Heartbeat connected on RolloutWorker_w1
+[2024-07-24 17:48:06,766][05152] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:08,330][05156] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:08,791][03928] Heartbeat connected on RolloutWorker_w5
+[2024-07-24 17:48:08,879][05154] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:08,885][05153] Decorrelating experience for 64 frames...
+[2024-07-24 17:48:09,400][03928] Heartbeat connected on RolloutWorker_w6
+[2024-07-24 17:48:11,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 101.9. Samples: 1528. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2024-07-24 17:48:11,392][03928] Avg episode reward: [(0, '3.311')]
+[2024-07-24 17:48:12,049][05149] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:12,260][05135] Signal inference workers to stop experience collection...
+[2024-07-24 17:48:12,282][05148] InferenceWorker_p0-w0: stopping experience collection
+[2024-07-24 17:48:12,285][05153] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:12,381][05152] Decorrelating experience for 96 frames...
+[2024-07-24 17:48:12,399][03928] Heartbeat connected on RolloutWorker_w0
+[2024-07-24 17:48:12,471][03928] Heartbeat connected on RolloutWorker_w4
+[2024-07-24 17:48:12,546][03928] Heartbeat connected on RolloutWorker_w3
+[2024-07-24 17:48:13,855][05135] Signal inference workers to resume experience collection...
+[2024-07-24 17:48:13,862][05148] InferenceWorker_p0-w0: resuming experience collection
+[2024-07-24 17:48:16,388][03928] Fps is (10 sec: 1228.8, 60 sec: 614.4, 300 sec: 614.4). Total num frames: 12288. Throughput: 0: 172.4. Samples: 3448. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
+[2024-07-24 17:48:16,392][03928] Avg episode reward: [(0, '3.234')]
+[2024-07-24 17:48:21,388][03928] Fps is (10 sec: 2867.2, 60 sec: 1146.9, 300 sec: 1146.9). Total num frames: 28672. Throughput: 0: 210.6. Samples: 5264. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:48:21,390][03928] Avg episode reward: [(0, '3.563')]
+[2024-07-24 17:48:24,277][05148] Updated weights for policy 0, policy_version 10 (0.0309)
+[2024-07-24 17:48:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 49152. Throughput: 0: 365.7. Samples: 10972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:48:26,394][03928] Avg episode reward: [(0, '4.026')]
+[2024-07-24 17:48:31,388][03928] Fps is (10 sec: 3686.4, 60 sec: 1872.5, 300 sec: 1872.5). Total num frames: 65536. Throughput: 0: 491.4. Samples: 17200. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:48:31,390][03928] Avg episode reward: [(0, '4.367')]
+[2024-07-24 17:48:36,394][03928] Fps is (10 sec: 2865.5, 60 sec: 1945.3, 300 sec: 1945.3). Total num frames: 77824. Throughput: 0: 476.4. Samples: 19060. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:48:36,396][03928] Avg episode reward: [(0, '4.399')]
+[2024-07-24 17:48:36,500][05148] Updated weights for policy 0, policy_version 20 (0.0046)
+[2024-07-24 17:48:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 2184.5, 300 sec: 2184.5). Total num frames: 98304. Throughput: 0: 522.8. Samples: 23528. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:48:41,390][03928] Avg episode reward: [(0, '4.341')]
+[2024-07-24 17:48:46,388][03928] Fps is (10 sec: 4098.5, 60 sec: 2375.7, 300 sec: 2375.7). Total num frames: 118784. Throughput: 0: 668.5. Samples: 30084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:48:46,390][03928] Avg episode reward: [(0, '4.325')]
+[2024-07-24 17:48:46,411][05135] Saving new best policy, reward=4.325!
+[2024-07-24 17:48:46,413][05148] Updated weights for policy 0, policy_version 30 (0.0034)
+[2024-07-24 17:48:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 135168. Throughput: 0: 729.8. Samples: 32840. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
+[2024-07-24 17:48:51,391][03928] Avg episode reward: [(0, '4.491')]
+[2024-07-24 17:48:51,404][05135] Saving new best policy, reward=4.491!
+[2024-07-24 17:48:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 2525.9, 300 sec: 2525.9). Total num frames: 151552. Throughput: 0: 784.5. Samples: 36830. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:48:56,393][03928] Avg episode reward: [(0, '4.468')]
+[2024-07-24 17:48:58,944][05148] Updated weights for policy 0, policy_version 40 (0.0043)
+[2024-07-24 17:49:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 2867.2, 300 sec: 2646.6). Total num frames: 172032. Throughput: 0: 882.2. Samples: 43146. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:49:01,394][03928] Avg episode reward: [(0, '4.428')]
+[2024-07-24 17:49:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3208.5, 300 sec: 2750.2). Total num frames: 192512. Throughput: 0: 912.7. Samples: 46336. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:49:06,393][03928] Avg episode reward: [(0, '4.409')]
+[2024-07-24 17:49:10,932][05148] Updated weights for policy 0, policy_version 50 (0.0017)
+[2024-07-24 17:49:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 2730.7). Total num frames: 204800. Throughput: 0: 884.5. Samples: 50774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:49:11,395][03928] Avg episode reward: [(0, '4.429')]
+[2024-07-24 17:49:16,389][03928] Fps is (10 sec: 3276.2, 60 sec: 3549.8, 300 sec: 2815.9). Total num frames: 225280. Throughput: 0: 862.8. Samples: 56028. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:49:16,392][03928] Avg episode reward: [(0, '4.423')]
+[2024-07-24 17:49:20,928][05148] Updated weights for policy 0, policy_version 60 (0.0029)
+[2024-07-24 17:49:21,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 2891.3). Total num frames: 245760. Throughput: 0: 894.8. Samples: 59320. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:49:21,390][03928] Avg episode reward: [(0, '4.223')]
+[2024-07-24 17:49:26,389][03928] Fps is (10 sec: 3277.0, 60 sec: 3481.5, 300 sec: 2867.2). Total num frames: 258048. Throughput: 0: 918.1. Samples: 64842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:49:26,394][03928] Avg episode reward: [(0, '4.287')]
+[2024-07-24 17:49:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 2888.8). Total num frames: 274432. Throughput: 0: 868.9. Samples: 69184. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:49:31,393][03928] Avg episode reward: [(0, '4.392')]
+[2024-07-24 17:49:31,401][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth...
+[2024-07-24 17:49:33,441][05148] Updated weights for policy 0, policy_version 70 (0.0034)
+[2024-07-24 17:49:36,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3686.8, 300 sec: 2990.1). Total num frames: 299008. Throughput: 0: 876.0. Samples: 72260. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:49:36,393][03928] Avg episode reward: [(0, '4.597')]
+[2024-07-24 17:49:36,398][05135] Saving new best policy, reward=4.597!
+[2024-07-24 17:49:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3003.7). Total num frames: 315392. Throughput: 0: 928.3. Samples: 78604. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2024-07-24 17:49:41,390][03928] Avg episode reward: [(0, '4.577')]
+[2024-07-24 17:49:45,697][05148] Updated weights for policy 0, policy_version 80 (0.0059)
+[2024-07-24 17:49:46,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 2978.9). Total num frames: 327680. Throughput: 0: 875.9. Samples: 82562. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2024-07-24 17:49:46,393][03928] Avg episode reward: [(0, '4.488')]
+[2024-07-24 17:49:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3027.5). Total num frames: 348160. Throughput: 0: 865.2. Samples: 85272. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:49:51,389][03928] Avg episode reward: [(0, '4.380')]
+[2024-07-24 17:49:55,556][05148] Updated weights for policy 0, policy_version 90 (0.0020)
+[2024-07-24 17:49:56,390][03928] Fps is (10 sec: 4095.1, 60 sec: 3618.0, 300 sec: 3071.9). Total num frames: 368640. Throughput: 0: 909.7. Samples: 91712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:49:56,395][03928] Avg episode reward: [(0, '4.437')]
+[2024-07-24 17:50:01,388][03928] Fps is (10 sec: 3686.2, 60 sec: 3549.8, 300 sec: 3080.2). Total num frames: 385024. Throughput: 0: 901.0. Samples: 96574. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:50:01,394][03928] Avg episode reward: [(0, '4.362')]
+[2024-07-24 17:50:06,388][03928] Fps is (10 sec: 3277.5, 60 sec: 3481.6, 300 sec: 3087.8). Total num frames: 401408. Throughput: 0: 872.7. Samples: 98592. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:50:06,390][03928] Avg episode reward: [(0, '4.471')]
+[2024-07-24 17:50:07,868][05148] Updated weights for policy 0, policy_version 100 (0.0030)
+[2024-07-24 17:50:11,388][03928] Fps is (10 sec: 3686.6, 60 sec: 3618.1, 300 sec: 3125.1). Total num frames: 421888. Throughput: 0: 887.2. Samples: 104764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:50:11,390][03928] Avg episode reward: [(0, '4.783')]
+[2024-07-24 17:50:11,402][05135] Saving new best policy, reward=4.783!
+[2024-07-24 17:50:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3130.5). Total num frames: 438272. Throughput: 0: 918.9. Samples: 110536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:50:16,390][03928] Avg episode reward: [(0, '4.747')]
+[2024-07-24 17:50:19,489][05148] Updated weights for policy 0, policy_version 110 (0.0022)
+[2024-07-24 17:50:21,389][03928] Fps is (10 sec: 3276.5, 60 sec: 3481.5, 300 sec: 3135.5). Total num frames: 454656. Throughput: 0: 893.6. Samples: 112474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:50:21,391][03928] Avg episode reward: [(0, '4.800')]
+[2024-07-24 17:50:21,402][05135] Saving new best policy, reward=4.800!
+[2024-07-24 17:50:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3167.6). Total num frames: 475136. Throughput: 0: 866.0. Samples: 117576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:50:26,394][03928] Avg episode reward: [(0, '4.561')]
+[2024-07-24 17:50:29,833][05148] Updated weights for policy 0, policy_version 120 (0.0026)
+[2024-07-24 17:50:31,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3686.4, 300 sec: 3197.5). Total num frames: 495616. Throughput: 0: 920.4. Samples: 123978. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:50:31,393][03928] Avg episode reward: [(0, '4.471')]
+[2024-07-24 17:50:36,392][03928] Fps is (10 sec: 3275.5, 60 sec: 3481.4, 300 sec: 3174.3). Total num frames: 507904. Throughput: 0: 914.1. Samples: 126410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:50:36,400][03928] Avg episode reward: [(0, '4.468')]
+[2024-07-24 17:50:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3202.3). Total num frames: 528384. Throughput: 0: 866.4. Samples: 130698. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:50:41,394][03928] Avg episode reward: [(0, '4.585')]
+[2024-07-24 17:50:42,178][05148] Updated weights for policy 0, policy_version 130 (0.0022)
+[2024-07-24 17:50:46,388][03928] Fps is (10 sec: 4097.7, 60 sec: 3686.4, 300 sec: 3228.6). Total num frames: 548864. Throughput: 0: 902.2. Samples: 137172. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:50:46,390][03928] Avg episode reward: [(0, '4.674')]
+[2024-07-24 17:50:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3230.0). Total num frames: 565248. Throughput: 0: 928.3. Samples: 140364. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:50:51,390][03928] Avg episode reward: [(0, '4.728')]
+[2024-07-24 17:50:53,745][05148] Updated weights for policy 0, policy_version 140 (0.0019)
+[2024-07-24 17:50:56,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.7, 300 sec: 3208.5). Total num frames: 577536. Throughput: 0: 883.2. Samples: 144510. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:50:56,391][03928] Avg episode reward: [(0, '4.697')]
+[2024-07-24 17:51:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3254.7). Total num frames: 602112. Throughput: 0: 885.9. Samples: 150400. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:51:01,390][03928] Avg episode reward: [(0, '4.561')]
+[2024-07-24 17:51:03,973][05148] Updated weights for policy 0, policy_version 150 (0.0021)
+[2024-07-24 17:51:06,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3276.8). Total num frames: 622592. Throughput: 0: 915.7. Samples: 153680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:51:06,393][03928] Avg episode reward: [(0, '4.519')]
+[2024-07-24 17:51:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3255.8). Total num frames: 634880. Throughput: 0: 916.7. Samples: 158826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:51:11,390][03928] Avg episode reward: [(0, '4.657')]
+[2024-07-24 17:51:16,289][05148] Updated weights for policy 0, policy_version 160 (0.0023)
+[2024-07-24 17:51:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3276.8). Total num frames: 655360. Throughput: 0: 877.6. Samples: 163470. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:51:16,389][03928] Avg episode reward: [(0, '4.728')]
+[2024-07-24 17:51:21,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3296.8). Total num frames: 675840. Throughput: 0: 894.7. Samples: 166666. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:51:21,390][03928] Avg episode reward: [(0, '4.468')]
+[2024-07-24 17:51:26,391][03928] Fps is (10 sec: 3685.3, 60 sec: 3617.9, 300 sec: 3296.3). Total num frames: 692224. Throughput: 0: 935.9. Samples: 172818. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:51:26,394][03928] Avg episode reward: [(0, '4.711')]
+[2024-07-24 17:51:27,256][05148] Updated weights for policy 0, policy_version 170 (0.0030)
+[2024-07-24 17:51:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3276.8). Total num frames: 704512. Throughput: 0: 881.2. Samples: 176826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:51:31,390][03928] Avg episode reward: [(0, '4.892')]
+[2024-07-24 17:51:31,477][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000173_708608.pth...
+[2024-07-24 17:51:31,599][05135] Saving new best policy, reward=4.892!
+[2024-07-24 17:51:36,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3686.7, 300 sec: 3314.0). Total num frames: 729088. Throughput: 0: 877.3. Samples: 179844. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:51:36,390][03928] Avg episode reward: [(0, '4.768')]
+[2024-07-24 17:51:38,085][05148] Updated weights for policy 0, policy_version 180 (0.0028)
+[2024-07-24 17:51:41,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3331.4). Total num frames: 749568. Throughput: 0: 929.3. Samples: 186330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:51:41,390][03928] Avg episode reward: [(0, '4.802')]
+[2024-07-24 17:51:46,391][03928] Fps is (10 sec: 3275.6, 60 sec: 3549.7, 300 sec: 3312.4). Total num frames: 761856. Throughput: 0: 899.0. Samples: 190858. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:51:46,394][03928] Avg episode reward: [(0, '4.912')]
+[2024-07-24 17:51:46,399][05135] Saving new best policy, reward=4.912!
+[2024-07-24 17:51:50,581][05148] Updated weights for policy 0, policy_version 190 (0.0026)
+[2024-07-24 17:51:51,388][03928] Fps is (10 sec: 2867.1, 60 sec: 3549.9, 300 sec: 3311.7). Total num frames: 778240. Throughput: 0: 871.0. Samples: 192876. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:51:51,394][03928] Avg episode reward: [(0, '5.131')]
+[2024-07-24 17:51:51,407][05135] Saving new best policy, reward=5.131!
+[2024-07-24 17:51:56,388][03928] Fps is (10 sec: 4097.5, 60 sec: 3754.7, 300 sec: 3345.1). Total num frames: 802816. Throughput: 0: 897.2. Samples: 199200. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:51:56,395][03928] Avg episode reward: [(0, '5.337')]
+[2024-07-24 17:51:56,399][05135] Saving new best policy, reward=5.337!
+[2024-07-24 17:52:01,224][05148] Updated weights for policy 0, policy_version 200 (0.0020)
+[2024-07-24 17:52:01,388][03928] Fps is (10 sec: 4096.1, 60 sec: 3618.1, 300 sec: 3343.7). Total num frames: 819200. Throughput: 0: 921.6. Samples: 204940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:52:01,391][03928] Avg episode reward: [(0, '5.275')]
+[2024-07-24 17:52:06,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3326.0). Total num frames: 831488. Throughput: 0: 892.3. Samples: 206818. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:52:06,394][03928] Avg episode reward: [(0, '5.096')]
+[2024-07-24 17:52:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3341.1). Total num frames: 851968. Throughput: 0: 882.4. Samples: 212524. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:52:11,390][03928] Avg episode reward: [(0, '5.151')]
+[2024-07-24 17:52:12,252][05148] Updated weights for policy 0, policy_version 210 (0.0053)
+[2024-07-24 17:52:16,391][03928] Fps is (10 sec: 4504.0, 60 sec: 3686.2, 300 sec: 3371.3). Total num frames: 876544. Throughput: 0: 934.8. Samples: 218894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:52:16,393][03928] Avg episode reward: [(0, '5.156')]
+[2024-07-24 17:52:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3354.1). Total num frames: 888832. Throughput: 0: 913.6. Samples: 220954. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:52:21,389][03928] Avg episode reward: [(0, '5.318')]
+[2024-07-24 17:52:24,641][05148] Updated weights for policy 0, policy_version 220 (0.0027)
+[2024-07-24 17:52:26,388][03928] Fps is (10 sec: 2868.2, 60 sec: 3550.0, 300 sec: 3352.7). Total num frames: 905216. Throughput: 0: 875.1. Samples: 225710. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:52:26,393][03928] Avg episode reward: [(0, '5.438')]
+[2024-07-24 17:52:26,398][05135] Saving new best policy, reward=5.438!
+[2024-07-24 17:52:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3336.4). Total num frames: 917504. Throughput: 0: 867.0. Samples: 229870. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:52:31,393][03928] Avg episode reward: [(0, '5.211')]
+[2024-07-24 17:52:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3335.3). Total num frames: 933888. Throughput: 0: 872.3. Samples: 232130. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:52:36,392][03928] Avg episode reward: [(0, '5.211')]
+[2024-07-24 17:52:39,228][05148] Updated weights for policy 0, policy_version 230 (0.0039)
+[2024-07-24 17:52:41,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3319.9). Total num frames: 946176. Throughput: 0: 821.2. Samples: 236156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:52:41,396][03928] Avg episode reward: [(0, '5.159')]
+[2024-07-24 17:52:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.5, 300 sec: 3333.3). Total num frames: 966656. Throughput: 0: 827.3. Samples: 242168. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:52:46,394][03928] Avg episode reward: [(0, '5.756')]
+[2024-07-24 17:52:46,416][05135] Saving new best policy, reward=5.756!
+[2024-07-24 17:52:49,369][05148] Updated weights for policy 0, policy_version 240 (0.0026)
+[2024-07-24 17:52:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3346.2). Total num frames: 987136. Throughput: 0: 854.5. Samples: 245272. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
+[2024-07-24 17:52:51,396][03928] Avg episode reward: [(0, '5.792')]
+[2024-07-24 17:52:51,415][05135] Saving new best policy, reward=5.792!
+[2024-07-24 17:52:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3387.9). Total num frames: 999424. Throughput: 0: 829.7. Samples: 249862. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
+[2024-07-24 17:52:56,391][03928] Avg episode reward: [(0, '5.909')]
+[2024-07-24 17:52:56,396][05135] Saving new best policy, reward=5.909!
+[2024-07-24 17:53:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3457.3). Total num frames: 1019904. Throughput: 0: 807.6. Samples: 255232. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:53:01,392][03928] Avg episode reward: [(0, '6.161')]
+[2024-07-24 17:53:01,402][05135] Saving new best policy, reward=6.161!
+[2024-07-24 17:53:01,642][05148] Updated weights for policy 0, policy_version 250 (0.0034)
+[2024-07-24 17:53:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1040384. Throughput: 0: 832.3. Samples: 258408. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:53:06,390][03928] Avg episode reward: [(0, '6.466')]
+[2024-07-24 17:53:06,425][05135] Saving new best policy, reward=6.466!
+[2024-07-24 17:53:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 1056768. Throughput: 0: 849.3. Samples: 263928. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:53:11,394][03928] Avg episode reward: [(0, '6.500')]
+[2024-07-24 17:53:11,403][05135] Saving new best policy, reward=6.500!
+[2024-07-24 17:53:13,856][05148] Updated weights for policy 0, policy_version 260 (0.0036)
+[2024-07-24 17:53:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3277.0, 300 sec: 3540.6). Total num frames: 1073152. Throughput: 0: 851.6. Samples: 268190. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:53:16,390][03928] Avg episode reward: [(0, '6.813')]
+[2024-07-24 17:53:16,397][05135] Saving new best policy, reward=6.813!
+[2024-07-24 17:53:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 1093632. Throughput: 0: 869.2. Samples: 271246. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:53:21,394][03928] Avg episode reward: [(0, '7.181')]
+[2024-07-24 17:53:21,407][05135] Saving new best policy, reward=7.181!
+[2024-07-24 17:53:23,601][05148] Updated weights for policy 0, policy_version 270 (0.0017)
+[2024-07-24 17:53:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1114112. Throughput: 0: 923.5. Samples: 277714. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0)
+[2024-07-24 17:53:26,393][03928] Avg episode reward: [(0, '7.707')]
+[2024-07-24 17:53:26,396][05135] Saving new best policy, reward=7.707!
+[2024-07-24 17:53:31,389][03928] Fps is (10 sec: 3276.4, 60 sec: 3481.5, 300 sec: 3554.6). Total num frames: 1126400. Throughput: 0: 877.7. Samples: 281666. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0)
+[2024-07-24 17:53:31,393][03928] Avg episode reward: [(0, '8.232')]
+[2024-07-24 17:53:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000275_1126400.pth...
+[2024-07-24 17:53:31,613][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth
+[2024-07-24 17:53:31,628][05135] Saving new best policy, reward=8.232!
+[2024-07-24 17:53:36,048][05148] Updated weights for policy 0, policy_version 280 (0.0025)
+[2024-07-24 17:53:36,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1146880. Throughput: 0: 867.8. Samples: 284322. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
+[2024-07-24 17:53:36,390][03928] Avg episode reward: [(0, '8.515')]
+[2024-07-24 17:53:36,395][05135] Saving new best policy, reward=8.515!
+[2024-07-24 17:53:41,388][03928] Fps is (10 sec: 4096.5, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1167360. Throughput: 0: 908.0. Samples: 290720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:53:41,393][03928] Avg episode reward: [(0, '7.801')]
+[2024-07-24 17:53:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 1179648. Throughput: 0: 896.3. Samples: 295564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:53:46,390][03928] Avg episode reward: [(0, '7.531')]
+[2024-07-24 17:53:48,309][05148] Updated weights for policy 0, policy_version 290 (0.0025)
+[2024-07-24 17:53:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1200128. Throughput: 0: 870.8. Samples: 297594. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:53:51,390][03928] Avg episode reward: [(0, '7.313')]
+[2024-07-24 17:53:56,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1220608. Throughput: 0: 885.0. Samples: 303754. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:53:56,392][03928] Avg episode reward: [(0, '7.883')]
+[2024-07-24 17:53:58,087][05148] Updated weights for policy 0, policy_version 300 (0.0020)
+[2024-07-24 17:54:01,391][03928] Fps is (10 sec: 3685.3, 60 sec: 3618.0, 300 sec: 3540.6). Total num frames: 1236992. Throughput: 0: 926.2. Samples: 309870. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2024-07-24 17:54:01,394][03928] Avg episode reward: [(0, '8.880')]
+[2024-07-24 17:54:01,404][05135] Saving new best policy, reward=8.880!
+[2024-07-24 17:54:06,388][03928] Fps is (10 sec: 2867.0, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 1249280. Throughput: 0: 900.1. Samples: 311750. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:54:06,394][03928] Avg episode reward: [(0, '9.189')]
+[2024-07-24 17:54:06,396][05135] Saving new best policy, reward=9.189!
+[2024-07-24 17:54:10,527][05148] Updated weights for policy 0, policy_version 310 (0.0031)
+[2024-07-24 17:54:11,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1273856. Throughput: 0: 872.4. Samples: 316970. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:54:11,389][03928] Avg episode reward: [(0, '9.557')]
+[2024-07-24 17:54:11,400][05135] Saving new best policy, reward=9.557!
+[2024-07-24 17:54:16,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1294336. Throughput: 0: 924.4. Samples: 323264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:54:16,391][03928] Avg episode reward: [(0, '8.574')]
+[2024-07-24 17:54:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1306624. Throughput: 0: 917.8. Samples: 325624. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:54:21,393][03928] Avg episode reward: [(0, '8.608')]
+[2024-07-24 17:54:22,140][05148] Updated weights for policy 0, policy_version 320 (0.0014)
+[2024-07-24 17:54:26,388][03928] Fps is (10 sec: 2867.4, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1323008. Throughput: 0: 874.8. Samples: 330084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:54:26,393][03928] Avg episode reward: [(0, '8.180')]
+[2024-07-24 17:54:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3554.5). Total num frames: 1347584. Throughput: 0: 910.0. Samples: 336512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:54:31,390][03928] Avg episode reward: [(0, '8.560')]
+[2024-07-24 17:54:32,143][05148] Updated weights for policy 0, policy_version 330 (0.0029)
+[2024-07-24 17:54:36,391][03928] Fps is (10 sec: 4094.7, 60 sec: 3617.9, 300 sec: 3554.5). Total num frames: 1363968. Throughput: 0: 939.2. Samples: 339862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:54:36,394][03928] Avg episode reward: [(0, '8.343')]
+[2024-07-24 17:54:41,389][03928] Fps is (10 sec: 2866.9, 60 sec: 3481.5, 300 sec: 3554.5). Total num frames: 1376256. Throughput: 0: 892.6. Samples: 343920. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:54:41,393][03928] Avg episode reward: [(0, '8.515')]
+[2024-07-24 17:54:44,407][05148] Updated weights for policy 0, policy_version 340 (0.0042)
+[2024-07-24 17:54:46,388][03928] Fps is (10 sec: 3687.6, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 1400832. Throughput: 0: 885.0. Samples: 349690. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:54:46,390][03928] Avg episode reward: [(0, '9.127')]
+[2024-07-24 17:54:51,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1417216. Throughput: 0: 915.6. Samples: 352952. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:54:51,390][03928] Avg episode reward: [(0, '9.707')]
+[2024-07-24 17:54:51,398][05135] Saving new best policy, reward=9.707!
+[2024-07-24 17:54:56,394][03928] Fps is (10 sec: 2865.2, 60 sec: 3481.2, 300 sec: 3540.5). Total num frames: 1429504. Throughput: 0: 907.2. Samples: 357802. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:54:56,401][03928] Avg episode reward: [(0, '9.535')]
+[2024-07-24 17:54:56,447][05148] Updated weights for policy 0, policy_version 350 (0.0027)
+[2024-07-24 17:55:01,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1449984. Throughput: 0: 880.8. Samples: 362900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:55:01,396][03928] Avg episode reward: [(0, '9.454')]
+[2024-07-24 17:55:06,301][05148] Updated weights for policy 0, policy_version 360 (0.0021)
+[2024-07-24 17:55:06,388][03928] Fps is (10 sec: 4508.7, 60 sec: 3754.7, 300 sec: 3568.4). Total num frames: 1474560. Throughput: 0: 899.9. Samples: 366120. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:55:06,394][03928] Avg episode reward: [(0, '9.840')]
+[2024-07-24 17:55:06,398][05135] Saving new best policy, reward=9.840!
+[2024-07-24 17:55:11,389][03928] Fps is (10 sec: 4096.3, 60 sec: 3618.0, 300 sec: 3568.4). Total num frames: 1490944. Throughput: 0: 929.8. Samples: 371928. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
+[2024-07-24 17:55:11,391][03928] Avg episode reward: [(0, '9.908')]
+[2024-07-24 17:55:11,405][05135] Saving new best policy, reward=9.908!
+[2024-07-24 17:55:16,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1503232. Throughput: 0: 879.0. Samples: 376068. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:16,390][03928] Avg episode reward: [(0, '10.292')]
+[2024-07-24 17:55:16,395][05135] Saving new best policy, reward=10.292!
+[2024-07-24 17:55:18,830][05148] Updated weights for policy 0, policy_version 370 (0.0026)
+[2024-07-24 17:55:21,388][03928] Fps is (10 sec: 3277.2, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1523712. Throughput: 0: 870.9. Samples: 379052. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:21,394][03928] Avg episode reward: [(0, '10.595')]
+[2024-07-24 17:55:21,406][05135] Saving new best policy, reward=10.595!
+[2024-07-24 17:55:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1544192. Throughput: 0: 920.2. Samples: 385330. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:26,393][03928] Avg episode reward: [(0, '10.916')]
+[2024-07-24 17:55:26,403][05135] Saving new best policy, reward=10.916!
+[2024-07-24 17:55:30,606][05148] Updated weights for policy 0, policy_version 380 (0.0021)
+[2024-07-24 17:55:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1556480. Throughput: 0: 883.7. Samples: 389458. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:31,391][03928] Avg episode reward: [(0, '11.562')]
+[2024-07-24 17:55:31,403][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000380_1556480.pth...
+[2024-07-24 17:55:31,618][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000173_708608.pth
+[2024-07-24 17:55:31,638][05135] Saving new best policy, reward=11.562!
+[2024-07-24 17:55:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.8, 300 sec: 3540.6). Total num frames: 1572864. Throughput: 0: 859.0. Samples: 391606. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:36,393][03928] Avg episode reward: [(0, '12.422')]
+[2024-07-24 17:55:36,397][05135] Saving new best policy, reward=12.422!
+[2024-07-24 17:55:41,254][05148] Updated weights for policy 0, policy_version 390 (0.0027)
+[2024-07-24 17:55:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3554.5). Total num frames: 1597440. Throughput: 0: 894.4. Samples: 398042. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:55:41,394][03928] Avg episode reward: [(0, '11.908')]
+[2024-07-24 17:55:46,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1613824. Throughput: 0: 898.8. Samples: 403342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:55:46,393][03928] Avg episode reward: [(0, '11.706')]
+[2024-07-24 17:55:51,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1626112. Throughput: 0: 871.0. Samples: 405316. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:55:51,389][03928] Avg episode reward: [(0, '11.442')]
+[2024-07-24 17:55:53,512][05148] Updated weights for policy 0, policy_version 400 (0.0028)
+[2024-07-24 17:55:56,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3686.8, 300 sec: 3554.5). Total num frames: 1650688. Throughput: 0: 874.7. Samples: 411288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:55:56,389][03928] Avg episode reward: [(0, '11.429')]
+[2024-07-24 17:56:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.3, 300 sec: 3540.6). Total num frames: 1667072. Throughput: 0: 924.8. Samples: 417684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:56:01,392][03928] Avg episode reward: [(0, '12.423')]
+[2024-07-24 17:56:01,398][05135] Saving new best policy, reward=12.423!
+[2024-07-24 17:56:04,712][05148] Updated weights for policy 0, policy_version 410 (0.0038)
+[2024-07-24 17:56:06,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1683456. Throughput: 0: 901.4. Samples: 419614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:06,397][03928] Avg episode reward: [(0, '13.423')]
+[2024-07-24 17:56:06,402][05135] Saving new best policy, reward=13.423!
+[2024-07-24 17:56:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1703936. Throughput: 0: 873.0. Samples: 424614. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:56:11,393][03928] Avg episode reward: [(0, '13.634')]
+[2024-07-24 17:56:11,404][05135] Saving new best policy, reward=13.634!
+[2024-07-24 17:56:15,038][05148] Updated weights for policy 0, policy_version 420 (0.0035)
+[2024-07-24 17:56:16,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1724416. Throughput: 0: 926.9. Samples: 431170. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:16,392][03928] Avg episode reward: [(0, '14.456')]
+[2024-07-24 17:56:16,398][05135] Saving new best policy, reward=14.456!
+[2024-07-24 17:56:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 1736704. Throughput: 0: 934.4. Samples: 433656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:21,393][03928] Avg episode reward: [(0, '14.105')]
+[2024-07-24 17:56:26,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1753088. Throughput: 0: 881.3. Samples: 437702. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:26,389][03928] Avg episode reward: [(0, '14.255')]
+[2024-07-24 17:56:27,441][05148] Updated weights for policy 0, policy_version 430 (0.0014)
+[2024-07-24 17:56:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1777664. Throughput: 0: 908.4. Samples: 444218. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:31,392][03928] Avg episode reward: [(0, '14.285')]
+[2024-07-24 17:56:36,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 1794048. Throughput: 0: 938.3. Samples: 447540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:56:36,394][03928] Avg episode reward: [(0, '12.840')]
+[2024-07-24 17:56:38,177][05148] Updated weights for policy 0, policy_version 440 (0.0020)
+[2024-07-24 17:56:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1810432. Throughput: 0: 904.4. Samples: 451986. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:56:41,395][03928] Avg episode reward: [(0, '12.887')]
+[2024-07-24 17:56:46,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1830912. Throughput: 0: 887.7. Samples: 457632. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:56:46,390][03928] Avg episode reward: [(0, '12.672')]
+[2024-07-24 17:56:48,924][05148] Updated weights for policy 0, policy_version 450 (0.0028)
+[2024-07-24 17:56:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3554.5). Total num frames: 1851392. Throughput: 0: 916.4. Samples: 460850. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:56:51,389][03928] Avg episode reward: [(0, '12.969')]
+[2024-07-24 17:56:56,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3549.7, 300 sec: 3540.6). Total num frames: 1863680. Throughput: 0: 920.4. Samples: 466036. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:56:56,392][03928] Avg episode reward: [(0, '14.355')]
+[2024-07-24 17:57:01,391][03928] Fps is (10 sec: 2456.9, 60 sec: 3481.4, 300 sec: 3540.6). Total num frames: 1875968. Throughput: 0: 846.8. Samples: 469280. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:57:01,398][03928] Avg episode reward: [(0, '14.263')]
+[2024-07-24 17:57:03,873][05148] Updated weights for policy 0, policy_version 460 (0.0027)
+[2024-07-24 17:57:06,388][03928] Fps is (10 sec: 2867.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1892352. Throughput: 0: 834.0. Samples: 471184. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:57:06,390][03928] Avg episode reward: [(0, '13.679')]
+[2024-07-24 17:57:11,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3481.6, 300 sec: 3512.9). Total num frames: 1912832. Throughput: 0: 890.7. Samples: 477782. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2024-07-24 17:57:11,391][03928] Avg episode reward: [(0, '13.150')]
+[2024-07-24 17:57:14,244][05148] Updated weights for policy 0, policy_version 470 (0.0028)
+[2024-07-24 17:57:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 1929216. Throughput: 0: 853.8. Samples: 482640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:57:16,393][03928] Avg episode reward: [(0, '12.833')]
+[2024-07-24 17:57:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1945600. Throughput: 0: 820.7. Samples: 484472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:57:21,390][03928] Avg episode reward: [(0, '13.420')]
+[2024-07-24 17:57:25,932][05148] Updated weights for policy 0, policy_version 480 (0.0035)
+[2024-07-24 17:57:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1966080. Throughput: 0: 858.6. Samples: 490622. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:57:26,395][03928] Avg episode reward: [(0, '14.727')]
+[2024-07-24 17:57:26,406][05135] Saving new best policy, reward=14.727!
+[2024-07-24 17:57:31,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 1982464. Throughput: 0: 860.8. Samples: 496368. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 17:57:31,391][03928] Avg episode reward: [(0, '15.248')]
+[2024-07-24 17:57:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000484_1982464.pth...
+[2024-07-24 17:57:31,562][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000275_1126400.pth
+[2024-07-24 17:57:31,579][05135] Saving new best policy, reward=15.248!
+[2024-07-24 17:57:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3554.5). Total num frames: 1994752. Throughput: 0: 831.4. Samples: 498264. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:57:36,394][03928] Avg episode reward: [(0, '16.611')]
+[2024-07-24 17:57:36,404][05135] Saving new best policy, reward=16.611!
+[2024-07-24 17:57:38,558][05148] Updated weights for policy 0, policy_version 490 (0.0040)
+[2024-07-24 17:57:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2015232. Throughput: 0: 827.1. Samples: 503254. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:57:41,394][03928] Avg episode reward: [(0, '17.272')]
+[2024-07-24 17:57:41,460][05135] Saving new best policy, reward=17.272!
+[2024-07-24 17:57:46,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2035712. Throughput: 0: 892.8. Samples: 509452. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:57:46,392][03928] Avg episode reward: [(0, '16.641')]
+[2024-07-24 17:57:49,443][05148] Updated weights for policy 0, policy_version 500 (0.0047)
+[2024-07-24 17:57:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3568.4). Total num frames: 2052096. Throughput: 0: 906.4. Samples: 511970. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:57:51,390][03928] Avg episode reward: [(0, '16.230')]
+[2024-07-24 17:57:56,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2068480. Throughput: 0: 847.0. Samples: 515898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:57:56,395][03928] Avg episode reward: [(0, '15.687')]
+[2024-07-24 17:58:01,143][05148] Updated weights for policy 0, policy_version 510 (0.0030)
+[2024-07-24 17:58:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3554.5). Total num frames: 2088960. Throughput: 0: 877.1. Samples: 522110. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:58:01,393][03928] Avg episode reward: [(0, '15.674')]
+[2024-07-24 17:58:06,388][03928] Fps is (10 sec: 3687.2, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2105344. Throughput: 0: 904.7. Samples: 525182. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:58:06,395][03928] Avg episode reward: [(0, '15.286')]
+[2024-07-24 17:58:11,388][03928] Fps is (10 sec: 2867.1, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 2117632. Throughput: 0: 858.7. Samples: 529266. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:58:11,391][03928] Avg episode reward: [(0, '16.167')]
+[2024-07-24 17:58:13,887][05148] Updated weights for policy 0, policy_version 520 (0.0038)
+[2024-07-24 17:58:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 2138112. Throughput: 0: 851.4. Samples: 534680. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:58:16,396][03928] Avg episode reward: [(0, '16.406')]
+[2024-07-24 17:58:21,388][03928] Fps is (10 sec: 4096.2, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2158592. Throughput: 0: 877.3. Samples: 537744. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:58:21,390][03928] Avg episode reward: [(0, '16.621')]
+[2024-07-24 17:58:25,217][05148] Updated weights for policy 0, policy_version 530 (0.0030)
+[2024-07-24 17:58:26,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 2170880. Throughput: 0: 879.4. Samples: 542826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:58:26,392][03928] Avg episode reward: [(0, '17.562')]
+[2024-07-24 17:58:26,397][05135] Saving new best policy, reward=17.562!
+[2024-07-24 17:58:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 2187264. Throughput: 0: 841.2. Samples: 547308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:58:31,390][03928] Avg episode reward: [(0, '17.960')]
+[2024-07-24 17:58:31,398][05135] Saving new best policy, reward=17.960!
+[2024-07-24 17:58:36,281][05148] Updated weights for policy 0, policy_version 540 (0.0021)
+[2024-07-24 17:58:36,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2211840. Throughput: 0: 854.3. Samples: 550414. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:58:36,390][03928] Avg episode reward: [(0, '17.697')]
+[2024-07-24 17:58:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2228224. Throughput: 0: 909.0. Samples: 556802. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:58:41,390][03928] Avg episode reward: [(0, '18.157')]
+[2024-07-24 17:58:41,397][05135] Saving new best policy, reward=18.157!
+[2024-07-24 17:58:46,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 2240512. Throughput: 0: 855.6. Samples: 560614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:58:46,391][03928] Avg episode reward: [(0, '17.741')]
+[2024-07-24 17:58:48,629][05148] Updated weights for policy 0, policy_version 550 (0.0046)
+[2024-07-24 17:58:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 2260992. Throughput: 0: 851.8. Samples: 563514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:58:51,389][03928] Avg episode reward: [(0, '18.106')]
+[2024-07-24 17:58:56,388][03928] Fps is (10 sec: 4095.9, 60 sec: 3550.0, 300 sec: 3540.6). Total num frames: 2281472. Throughput: 0: 902.8. Samples: 569892. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:58:56,390][03928] Avg episode reward: [(0, '19.165')]
+[2024-07-24 17:58:56,394][05135] Saving new best policy, reward=19.165!
+[2024-07-24 17:58:59,781][05148] Updated weights for policy 0, policy_version 560 (0.0018)
+[2024-07-24 17:59:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 2297856. Throughput: 0: 886.2. Samples: 574558. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:59:01,394][03928] Avg episode reward: [(0, '18.524')]
+[2024-07-24 17:59:06,388][03928] Fps is (10 sec: 3276.9, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 2314240. Throughput: 0: 863.5. Samples: 576602. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:59:06,391][03928] Avg episode reward: [(0, '18.026')]
+[2024-07-24 17:59:10,396][05148] Updated weights for policy 0, policy_version 570 (0.0017)
+[2024-07-24 17:59:11,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2338816. Throughput: 0: 896.2. Samples: 583154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:59:11,390][03928] Avg episode reward: [(0, '17.310')]
+[2024-07-24 17:59:16,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2355200. Throughput: 0: 925.7. Samples: 588966. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:59:16,392][03928] Avg episode reward: [(0, '16.380')]
+[2024-07-24 17:59:21,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 2367488. Throughput: 0: 901.2. Samples: 590968. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 17:59:21,392][03928] Avg episode reward: [(0, '16.258')]
+[2024-07-24 17:59:22,734][05148] Updated weights for policy 0, policy_version 580 (0.0021)
+[2024-07-24 17:59:26,388][03928] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2387968. Throughput: 0: 883.1. Samples: 596542. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:59:26,394][03928] Avg episode reward: [(0, '16.607')]
+[2024-07-24 17:59:31,389][03928] Fps is (10 sec: 4504.9, 60 sec: 3754.6, 300 sec: 3554.5). Total num frames: 2412544. Throughput: 0: 941.3. Samples: 602974. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:59:31,392][03928] Avg episode reward: [(0, '17.820')]
+[2024-07-24 17:59:31,404][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000589_2412544.pth...
+[2024-07-24 17:59:31,552][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000380_1556480.pth
+[2024-07-24 17:59:32,709][05148] Updated weights for policy 0, policy_version 590 (0.0018)
+[2024-07-24 17:59:36,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.8, 300 sec: 3554.5). Total num frames: 2424832. Throughput: 0: 924.2. Samples: 605102. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 17:59:36,396][03928] Avg episode reward: [(0, '18.054')]
+[2024-07-24 17:59:41,388][03928] Fps is (10 sec: 2867.6, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 2441216. Throughput: 0: 884.9. Samples: 609712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 17:59:41,393][03928] Avg episode reward: [(0, '18.196')]
+[2024-07-24 17:59:44,452][05148] Updated weights for policy 0, policy_version 600 (0.0020)
+[2024-07-24 17:59:46,391][03928] Fps is (10 sec: 4094.9, 60 sec: 3754.5, 300 sec: 3554.5). Total num frames: 2465792. Throughput: 0: 922.6. Samples: 616076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 17:59:46,393][03928] Avg episode reward: [(0, '18.771')]
+[2024-07-24 17:59:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3568.5). Total num frames: 2482176. Throughput: 0: 946.2. Samples: 619182. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 17:59:51,390][03928] Avg episode reward: [(0, '18.468')]
+[2024-07-24 17:59:56,388][03928] Fps is (10 sec: 2868.1, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2494464. Throughput: 0: 890.5. Samples: 623228. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 17:59:56,389][03928] Avg episode reward: [(0, '18.800')]
+[2024-07-24 17:59:56,931][05148] Updated weights for policy 0, policy_version 610 (0.0034)
+[2024-07-24 18:00:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2519040. Throughput: 0: 895.3. Samples: 629256. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:00:01,396][03928] Avg episode reward: [(0, '19.406')]
+[2024-07-24 18:00:01,405][05135] Saving new best policy, reward=19.406!
+[2024-07-24 18:00:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2535424. Throughput: 0: 923.1. Samples: 632506. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:00:06,392][03928] Avg episode reward: [(0, '19.363')]
+[2024-07-24 18:00:06,694][05148] Updated weights for policy 0, policy_version 620 (0.0042)
+[2024-07-24 18:00:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2551808. Throughput: 0: 909.1. Samples: 637452. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2024-07-24 18:00:11,397][03928] Avg episode reward: [(0, '19.925')]
+[2024-07-24 18:00:11,422][05135] Saving new best policy, reward=19.925!
+[2024-07-24 18:00:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2568192. Throughput: 0: 878.8. Samples: 642518. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 18:00:16,394][03928] Avg episode reward: [(0, '20.714')]
+[2024-07-24 18:00:16,397][05135] Saving new best policy, reward=20.714!
+[2024-07-24 18:00:18,457][05148] Updated weights for policy 0, policy_version 630 (0.0045)
+[2024-07-24 18:00:21,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3686.3, 300 sec: 3540.6). Total num frames: 2588672. Throughput: 0: 902.1. Samples: 645700. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:00:21,395][03928] Avg episode reward: [(0, '20.538')]
+[2024-07-24 18:00:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 2605056. Throughput: 0: 927.8. Samples: 651462. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 18:00:26,390][03928] Avg episode reward: [(0, '19.510')]
+[2024-07-24 18:00:30,605][05148] Updated weights for policy 0, policy_version 640 (0.0027)
+[2024-07-24 18:00:31,388][03928] Fps is (10 sec: 3277.4, 60 sec: 3481.7, 300 sec: 3554.5). Total num frames: 2621440. Throughput: 0: 880.0. Samples: 655674. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:00:31,392][03928] Avg episode reward: [(0, '18.659')]
+[2024-07-24 18:00:36,390][03928] Fps is (10 sec: 4095.0, 60 sec: 3686.3, 300 sec: 3554.5). Total num frames: 2646016. Throughput: 0: 880.8. Samples: 658820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:00:36,394][03928] Avg episode reward: [(0, '18.538')]
+[2024-07-24 18:00:40,230][05148] Updated weights for policy 0, policy_version 650 (0.0023)
+[2024-07-24 18:00:41,394][03928] Fps is (10 sec: 4093.5, 60 sec: 3686.0, 300 sec: 3554.4). Total num frames: 2662400. Throughput: 0: 937.3. Samples: 665412. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 18:00:41,396][03928] Avg episode reward: [(0, '18.118')]
+[2024-07-24 18:00:46,388][03928] Fps is (10 sec: 2867.9, 60 sec: 3481.8, 300 sec: 3554.5). Total num frames: 2674688. Throughput: 0: 896.3. Samples: 669590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 18:00:46,393][03928] Avg episode reward: [(0, '19.090')]
+[2024-07-24 18:00:51,388][03928] Fps is (10 sec: 3278.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2695168. Throughput: 0: 879.9. Samples: 672100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:00:51,390][03928] Avg episode reward: [(0, '19.696')]
+[2024-07-24 18:00:52,577][05148] Updated weights for policy 0, policy_version 660 (0.0032)
+[2024-07-24 18:00:56,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 2715648. Throughput: 0: 911.4. Samples: 678464. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:00:56,392][03928] Avg episode reward: [(0, '19.752')]
+[2024-07-24 18:01:01,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3549.7, 300 sec: 3554.5). Total num frames: 2732032. Throughput: 0: 908.1. Samples: 683384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:01:01,396][03928] Avg episode reward: [(0, '20.074')]
+[2024-07-24 18:01:05,368][05148] Updated weights for policy 0, policy_version 670 (0.0046)
+[2024-07-24 18:01:06,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2748416. Throughput: 0: 880.1. Samples: 685302. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:01:06,391][03928] Avg episode reward: [(0, '19.223')]
+[2024-07-24 18:01:11,388][03928] Fps is (10 sec: 3687.1, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2768896. Throughput: 0: 880.8. Samples: 691096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:01:11,394][03928] Avg episode reward: [(0, '19.820')]
+[2024-07-24 18:01:14,889][05148] Updated weights for policy 0, policy_version 680 (0.0022)
+[2024-07-24 18:01:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2785280. Throughput: 0: 922.0. Samples: 697166. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:01:16,390][03928] Avg episode reward: [(0, '19.437')]
+[2024-07-24 18:01:21,389][03928] Fps is (10 sec: 3276.5, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2801664. Throughput: 0: 895.1. Samples: 699096. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 18:01:21,391][03928] Avg episode reward: [(0, '19.917')]
+[2024-07-24 18:01:26,388][03928] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 2809856. Throughput: 0: 828.2. Samples: 702678. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:01:26,389][03928] Avg episode reward: [(0, '20.748')]
+[2024-07-24 18:01:26,407][05135] Saving new best policy, reward=20.748!
+[2024-07-24 18:01:30,526][05148] Updated weights for policy 0, policy_version 690 (0.0020)
+[2024-07-24 18:01:31,388][03928] Fps is (10 sec: 2457.9, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 2826240. Throughput: 0: 832.5. Samples: 707054. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:01:31,390][03928] Avg episode reward: [(0, '20.377')]
+[2024-07-24 18:01:31,403][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000690_2826240.pth...
+[2024-07-24 18:01:31,542][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000484_1982464.pth
+[2024-07-24 18:01:36,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3276.8, 300 sec: 3498.9). Total num frames: 2842624. Throughput: 0: 835.9. Samples: 709716. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:01:36,393][03928] Avg episode reward: [(0, '19.809')]
+[2024-07-24 18:01:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3277.1, 300 sec: 3485.1). Total num frames: 2859008. Throughput: 0: 782.1. Samples: 713660. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 18:01:41,390][03928] Avg episode reward: [(0, '20.442')]
+[2024-07-24 18:01:43,036][05148] Updated weights for policy 0, policy_version 700 (0.0034)
+[2024-07-24 18:01:46,388][03928] Fps is (10 sec: 3687.1, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 2879488. Throughput: 0: 809.3. Samples: 719802. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:01:46,390][03928] Avg episode reward: [(0, '20.249')]
+[2024-07-24 18:01:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3499.0). Total num frames: 2895872. Throughput: 0: 835.3. Samples: 722892. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:01:51,392][03928] Avg episode reward: [(0, '20.259')]
+[2024-07-24 18:01:54,732][05148] Updated weights for policy 0, policy_version 710 (0.0032)
+[2024-07-24 18:01:56,388][03928] Fps is (10 sec: 3276.6, 60 sec: 3276.8, 300 sec: 3512.9). Total num frames: 2912256. Throughput: 0: 808.9. Samples: 727498. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:01:56,393][03928] Avg episode reward: [(0, '19.961')]
+[2024-07-24 18:02:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.9, 300 sec: 3512.8). Total num frames: 2928640. Throughput: 0: 785.3. Samples: 732506. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:01,394][03928] Avg episode reward: [(0, '20.125')]
+[2024-07-24 18:02:05,411][05148] Updated weights for policy 0, policy_version 720 (0.0025)
+[2024-07-24 18:02:06,390][03928] Fps is (10 sec: 4095.4, 60 sec: 3413.2, 300 sec: 3526.7). Total num frames: 2953216. Throughput: 0: 814.5. Samples: 735750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:06,394][03928] Avg episode reward: [(0, '19.647')]
+[2024-07-24 18:02:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 2965504. Throughput: 0: 861.1. Samples: 741426. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:02:11,394][03928] Avg episode reward: [(0, '19.592')]
+[2024-07-24 18:02:16,388][03928] Fps is (10 sec: 2867.8, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 2981888. Throughput: 0: 857.5. Samples: 745640. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:02:16,390][03928] Avg episode reward: [(0, '19.007')]
+[2024-07-24 18:02:17,820][05148] Updated weights for policy 0, policy_version 730 (0.0023)
+[2024-07-24 18:02:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3512.8). Total num frames: 3002368. Throughput: 0: 869.4. Samples: 748836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:21,390][03928] Avg episode reward: [(0, '18.736')]
+[2024-07-24 18:02:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 3022848. Throughput: 0: 920.6. Samples: 755088. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:26,390][03928] Avg episode reward: [(0, '18.715')]
+[2024-07-24 18:02:29,368][05148] Updated weights for policy 0, policy_version 740 (0.0024)
+[2024-07-24 18:02:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 3035136. Throughput: 0: 874.5. Samples: 759156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:02:31,394][03928] Avg episode reward: [(0, '19.211')]
+[2024-07-24 18:02:36,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3526.7). Total num frames: 3055616. Throughput: 0: 861.5. Samples: 761658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:36,391][03928] Avg episode reward: [(0, '18.984')]
+[2024-07-24 18:02:40,026][05148] Updated weights for policy 0, policy_version 750 (0.0042)
+[2024-07-24 18:02:41,389][03928] Fps is (10 sec: 4095.5, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 3076096. Throughput: 0: 898.7. Samples: 767940. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:02:41,392][03928] Avg episode reward: [(0, '20.207')]
+[2024-07-24 18:02:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3088384. Throughput: 0: 896.0. Samples: 772824. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:02:46,390][03928] Avg episode reward: [(0, '21.139')]
+[2024-07-24 18:02:46,392][05135] Saving new best policy, reward=21.139!
+[2024-07-24 18:02:51,388][03928] Fps is (10 sec: 2867.6, 60 sec: 3481.6, 300 sec: 3512.9). Total num frames: 3104768. Throughput: 0: 866.0. Samples: 774716. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:02:51,390][03928] Avg episode reward: [(0, '21.132')]
+[2024-07-24 18:02:52,666][05148] Updated weights for policy 0, policy_version 760 (0.0029)
+[2024-07-24 18:02:56,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3125248. Throughput: 0: 868.5. Samples: 780508. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:02:56,395][03928] Avg episode reward: [(0, '19.634')]
+[2024-07-24 18:03:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3141632. Throughput: 0: 900.7. Samples: 786170. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:01,390][03928] Avg episode reward: [(0, '19.616')]
+[2024-07-24 18:03:04,842][05148] Updated weights for policy 0, policy_version 770 (0.0015)
+[2024-07-24 18:03:06,391][03928] Fps is (10 sec: 3275.6, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 3158016. Throughput: 0: 871.3. Samples: 788048. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:06,394][03928] Avg episode reward: [(0, '19.844')]
+[2024-07-24 18:03:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3174400. Throughput: 0: 845.4. Samples: 793130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:03:11,390][03928] Avg episode reward: [(0, '18.447')]
+[2024-07-24 18:03:15,652][05148] Updated weights for policy 0, policy_version 780 (0.0026)
+[2024-07-24 18:03:16,388][03928] Fps is (10 sec: 3687.7, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3194880. Throughput: 0: 886.7. Samples: 799058. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:16,390][03928] Avg episode reward: [(0, '18.362')]
+[2024-07-24 18:03:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3512.8). Total num frames: 3207168. Throughput: 0: 884.3. Samples: 801450. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2024-07-24 18:03:21,393][03928] Avg episode reward: [(0, '19.209')]
+[2024-07-24 18:03:26,388][03928] Fps is (10 sec: 2867.0, 60 sec: 3345.0, 300 sec: 3512.8). Total num frames: 3223552. Throughput: 0: 833.3. Samples: 805440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:26,392][03928] Avg episode reward: [(0, '19.174')]
+[2024-07-24 18:03:28,459][05148] Updated weights for policy 0, policy_version 790 (0.0018)
+[2024-07-24 18:03:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3248128. Throughput: 0: 863.7. Samples: 811692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:31,390][03928] Avg episode reward: [(0, '19.706')]
+[2024-07-24 18:03:31,400][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000793_3248128.pth...
+[2024-07-24 18:03:31,534][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000589_2412544.pth
+[2024-07-24 18:03:36,388][03928] Fps is (10 sec: 4096.2, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3264512. Throughput: 0: 890.6. Samples: 814794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:03:36,392][03928] Avg episode reward: [(0, '19.738')]
+[2024-07-24 18:03:40,798][05148] Updated weights for policy 0, policy_version 800 (0.0048)
+[2024-07-24 18:03:41,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3512.8). Total num frames: 3276800. Throughput: 0: 854.3. Samples: 818950. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:03:41,395][03928] Avg episode reward: [(0, '19.912')]
+[2024-07-24 18:03:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3297280. Throughput: 0: 851.6. Samples: 824490. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:46,391][03928] Avg episode reward: [(0, '20.931')]
+[2024-07-24 18:03:50,775][05148] Updated weights for policy 0, policy_version 810 (0.0021)
+[2024-07-24 18:03:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3317760. Throughput: 0: 878.8. Samples: 827590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2024-07-24 18:03:51,390][03928] Avg episode reward: [(0, '20.755')]
+[2024-07-24 18:03:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3330048. Throughput: 0: 881.3. Samples: 832788. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:03:56,390][03928] Avg episode reward: [(0, '20.034')]
+[2024-07-24 18:04:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3350528. Throughput: 0: 847.3. Samples: 837188. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:04:01,394][03928] Avg episode reward: [(0, '20.330')]
+[2024-07-24 18:04:03,276][05148] Updated weights for policy 0, policy_version 820 (0.0017)
+[2024-07-24 18:04:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3550.1, 300 sec: 3499.0). Total num frames: 3371008. Throughput: 0: 866.7. Samples: 840450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:04:06,395][03928] Avg episode reward: [(0, '20.146')]
+[2024-07-24 18:04:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 3387392. Throughput: 0: 917.3. Samples: 846716. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:04:11,390][03928] Avg episode reward: [(0, '19.309')]
+[2024-07-24 18:04:15,446][05148] Updated weights for policy 0, policy_version 830 (0.0024)
+[2024-07-24 18:04:16,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3399680. Throughput: 0: 863.9. Samples: 850566. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:04:16,390][03928] Avg episode reward: [(0, '18.963')]
+[2024-07-24 18:04:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 3420160. Throughput: 0: 856.6. Samples: 853342. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:04:21,395][03928] Avg episode reward: [(0, '19.258')]
+[2024-07-24 18:04:25,600][05148] Updated weights for policy 0, policy_version 840 (0.0032)
+[2024-07-24 18:04:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3485.1). Total num frames: 3440640. Throughput: 0: 902.3. Samples: 859552. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:04:26,392][03928] Avg episode reward: [(0, '19.035')]
+[2024-07-24 18:04:31,393][03928] Fps is (10 sec: 3275.2, 60 sec: 3413.1, 300 sec: 3485.0). Total num frames: 3452928. Throughput: 0: 874.9. Samples: 863866. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:04:31,401][03928] Avg episode reward: [(0, '20.335')]
+[2024-07-24 18:04:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 3469312. Throughput: 0: 849.3. Samples: 865810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:04:36,395][03928] Avg episode reward: [(0, '21.311')]
+[2024-07-24 18:04:36,398][05135] Saving new best policy, reward=21.311!
+[2024-07-24 18:04:38,586][05148] Updated weights for policy 0, policy_version 850 (0.0018)
+[2024-07-24 18:04:41,388][03928] Fps is (10 sec: 3688.3, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 3489792. Throughput: 0: 867.2. Samples: 871812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
+[2024-07-24 18:04:41,390][03928] Avg episode reward: [(0, '22.288')]
+[2024-07-24 18:04:41,400][05135] Saving new best policy, reward=22.288!
+[2024-07-24 18:04:46,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3481.5, 300 sec: 3471.2). Total num frames: 3506176. Throughput: 0: 891.2. Samples: 877294. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:04:46,394][03928] Avg episode reward: [(0, '22.407')]
+[2024-07-24 18:04:46,401][05135] Saving new best policy, reward=22.407!
+[2024-07-24 18:04:51,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3471.2). Total num frames: 3518464. Throughput: 0: 858.4. Samples: 879080. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:04:51,390][03928] Avg episode reward: [(0, '22.836')]
+[2024-07-24 18:04:51,406][05135] Saving new best policy, reward=22.836!
+[2024-07-24 18:04:51,680][05148] Updated weights for policy 0, policy_version 860 (0.0018)
+[2024-07-24 18:04:56,388][03928] Fps is (10 sec: 3277.5, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 3538944. Throughput: 0: 835.6. Samples: 884318. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:04:56,393][03928] Avg episode reward: [(0, '23.839')]
+[2024-07-24 18:04:56,399][05135] Saving new best policy, reward=23.839!
+[2024-07-24 18:05:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3559424. Throughput: 0: 885.5. Samples: 890412. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:05:01,395][03928] Avg episode reward: [(0, '21.871')]
+[2024-07-24 18:05:01,522][05148] Updated weights for policy 0, policy_version 870 (0.0035)
+[2024-07-24 18:05:06,388][03928] Fps is (10 sec: 3686.2, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3575808. Throughput: 0: 874.6. Samples: 892698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:05:06,398][03928] Avg episode reward: [(0, '21.223')]
+[2024-07-24 18:05:11,391][03928] Fps is (10 sec: 3275.7, 60 sec: 3413.1, 300 sec: 3471.1). Total num frames: 3592192. Throughput: 0: 835.0. Samples: 897130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:05:11,399][03928] Avg episode reward: [(0, '21.649')]
+[2024-07-24 18:05:13,694][05148] Updated weights for policy 0, policy_version 880 (0.0056)
+[2024-07-24 18:05:16,388][03928] Fps is (10 sec: 3686.5, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 3612672. Throughput: 0: 884.1. Samples: 903646. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:16,396][03928] Avg episode reward: [(0, '19.774')]
+[2024-07-24 18:05:21,388][03928] Fps is (10 sec: 3687.6, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3629056. Throughput: 0: 912.9. Samples: 906890. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 18:05:21,393][03928] Avg episode reward: [(0, '20.340')]
+[2024-07-24 18:05:26,307][05148] Updated weights for policy 0, policy_version 890 (0.0022)
+[2024-07-24 18:05:26,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3645440. Throughput: 0: 862.6. Samples: 910628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:26,391][03928] Avg episode reward: [(0, '20.521')]
+[2024-07-24 18:05:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.9, 300 sec: 3443.4). Total num frames: 3661824. Throughput: 0: 859.8. Samples: 915984. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:31,390][03928] Avg episode reward: [(0, '20.443')]
+[2024-07-24 18:05:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000895_3665920.pth...
+[2024-07-24 18:05:31,529][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000690_2826240.pth
+[2024-07-24 18:05:36,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3457.4). Total num frames: 3682304. Throughput: 0: 885.6. Samples: 918930. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:36,390][03928] Avg episode reward: [(0, '21.409')]
+[2024-07-24 18:05:36,714][05148] Updated weights for policy 0, policy_version 900 (0.0019)
+[2024-07-24 18:05:41,391][03928] Fps is (10 sec: 3275.8, 60 sec: 3413.2, 300 sec: 3457.3). Total num frames: 3694592. Throughput: 0: 879.7. Samples: 923906. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:41,393][03928] Avg episode reward: [(0, '21.583')]
+[2024-07-24 18:05:46,388][03928] Fps is (10 sec: 3276.7, 60 sec: 3481.7, 300 sec: 3457.3). Total num frames: 3715072. Throughput: 0: 852.8. Samples: 928790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2024-07-24 18:05:46,391][03928] Avg episode reward: [(0, '22.659')]
+[2024-07-24 18:05:50,046][05148] Updated weights for policy 0, policy_version 910 (0.0027)
+[2024-07-24 18:05:51,388][03928] Fps is (10 sec: 3277.7, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3727360. Throughput: 0: 858.8. Samples: 931342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:51,393][03928] Avg episode reward: [(0, '22.964')]
+[2024-07-24 18:05:56,391][03928] Fps is (10 sec: 2456.7, 60 sec: 3344.9, 300 sec: 3415.6). Total num frames: 3739648. Throughput: 0: 846.9. Samples: 935240. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:05:56,394][03928] Avg episode reward: [(0, '21.320')]
+[2024-07-24 18:06:01,388][03928] Fps is (10 sec: 2867.3, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 3756032. Throughput: 0: 786.0. Samples: 939016. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:06:01,394][03928] Avg episode reward: [(0, '21.737')]
+[2024-07-24 18:06:04,144][05148] Updated weights for policy 0, policy_version 920 (0.0029)
+[2024-07-24 18:06:06,388][03928] Fps is (10 sec: 3687.8, 60 sec: 3345.1, 300 sec: 3415.6). Total num frames: 3776512. Throughput: 0: 778.0. Samples: 941898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:06:06,390][03928] Avg episode reward: [(0, '21.229')]
+[2024-07-24 18:06:11,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.5, 300 sec: 3429.5). Total num frames: 3796992. Throughput: 0: 837.8. Samples: 948330. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:06:11,393][03928] Avg episode reward: [(0, '20.337')]
+[2024-07-24 18:06:15,098][05148] Updated weights for policy 0, policy_version 930 (0.0019)
+[2024-07-24 18:06:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3415.7). Total num frames: 3809280. Throughput: 0: 824.0. Samples: 953066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:06:16,393][03928] Avg episode reward: [(0, '19.589')]
+[2024-07-24 18:06:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3457.3). Total num frames: 3829760. Throughput: 0: 804.8. Samples: 955148. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:06:21,392][03928] Avg episode reward: [(0, '18.518')]
+[2024-07-24 18:06:26,161][05148] Updated weights for policy 0, policy_version 940 (0.0024)
+[2024-07-24 18:06:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3850240. Throughput: 0: 833.7. Samples: 961420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:06:26,395][03928] Avg episode reward: [(0, '19.677')]
+[2024-07-24 18:06:31,388][03928] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3866624. Throughput: 0: 850.7. Samples: 967072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2024-07-24 18:06:31,394][03928] Avg episode reward: [(0, '20.345')]
+[2024-07-24 18:06:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3457.3). Total num frames: 3878912. Throughput: 0: 838.0. Samples: 969050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:06:36,394][03928] Avg episode reward: [(0, '19.104')]
+[2024-07-24 18:06:38,424][05148] Updated weights for policy 0, policy_version 950 (0.0020)
+[2024-07-24 18:06:41,389][03928] Fps is (10 sec: 3686.1, 60 sec: 3481.7, 300 sec: 3471.2). Total num frames: 3903488. Throughput: 0: 874.1. Samples: 974570. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:06:41,396][03928] Avg episode reward: [(0, '20.782')]
+[2024-07-24 18:06:46,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 3923968. Throughput: 0: 937.2. Samples: 981192. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:06:46,391][03928] Avg episode reward: [(0, '21.816')]
+[2024-07-24 18:06:48,710][05148] Updated weights for policy 0, policy_version 960 (0.0034)
+[2024-07-24 18:06:51,388][03928] Fps is (10 sec: 3277.2, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3936256. Throughput: 0: 921.8. Samples: 983378. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:06:51,392][03928] Avg episode reward: [(0, '21.747')]
+[2024-07-24 18:06:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.4, 300 sec: 3485.1). Total num frames: 3956736. Throughput: 0: 878.6. Samples: 987866. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2024-07-24 18:06:56,390][03928] Avg episode reward: [(0, '21.512')]
+[2024-07-24 18:07:00,142][05148] Updated weights for policy 0, policy_version 970 (0.0033)
+[2024-07-24 18:07:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3471.2). Total num frames: 3977216. Throughput: 0: 915.2. Samples: 994248. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2024-07-24 18:07:01,390][03928] Avg episode reward: [(0, '22.009')]
+[2024-07-24 18:07:06,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3485.1). Total num frames: 3993600. Throughput: 0: 937.6. Samples: 997340. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2024-07-24 18:07:06,393][03928] Avg episode reward: [(0, '21.690')]
+[2024-07-24 18:07:10,532][05135] Stopping Batcher_0...
+[2024-07-24 18:07:10,533][05135] Loop batcher_evt_loop terminating...
+[2024-07-24 18:07:10,535][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2024-07-24 18:07:10,547][03928] Component Batcher_0 stopped!
+[2024-07-24 18:07:10,629][03928] Component RolloutWorker_w6 stopped!
+[2024-07-24 18:07:10,629][05154] Stopping RolloutWorker_w6...
+[2024-07-24 18:07:10,666][03928] Component RolloutWorker_w0 stopped!
+[2024-07-24 18:07:10,652][05154] Loop rollout_proc6_evt_loop terminating...
+[2024-07-24 18:07:10,670][05148] Weights refcount: 2 0
+[2024-07-24 18:07:10,665][05149] Stopping RolloutWorker_w0...
+[2024-07-24 18:07:10,676][05149] Loop rollout_proc0_evt_loop terminating...
+[2024-07-24 18:07:10,681][03928] Component InferenceWorker_p0-w0 stopped!
+[2024-07-24 18:07:10,684][03928] Component RolloutWorker_w3 stopped!
+[2024-07-24 18:07:10,681][05152] Stopping RolloutWorker_w3...
+[2024-07-24 18:07:10,687][05152] Loop rollout_proc3_evt_loop terminating...
+[2024-07-24 18:07:10,688][05148] Stopping InferenceWorker_p0-w0...
+[2024-07-24 18:07:10,689][05148] Loop inference_proc0-0_evt_loop terminating...
+[2024-07-24 18:07:10,696][03928] Component RolloutWorker_w7 stopped!
+[2024-07-24 18:07:10,699][05155] Stopping RolloutWorker_w7...
+[2024-07-24 18:07:10,714][03928] Component RolloutWorker_w4 stopped!
+[2024-07-24 18:07:10,703][05155] Loop rollout_proc7_evt_loop terminating...
+[2024-07-24 18:07:10,714][05153] Stopping RolloutWorker_w4...
+[2024-07-24 18:07:10,726][05150] Stopping RolloutWorker_w1...
+[2024-07-24 18:07:10,722][03928] Component RolloutWorker_w1 stopped!
+[2024-07-24 18:07:10,730][05150] Loop rollout_proc1_evt_loop terminating...
+[2024-07-24 18:07:10,721][05153] Loop rollout_proc4_evt_loop terminating...
+[2024-07-24 18:07:10,733][03928] Component RolloutWorker_w5 stopped!
+[2024-07-24 18:07:10,733][05156] Stopping RolloutWorker_w5...
+[2024-07-24 18:07:10,740][05156] Loop rollout_proc5_evt_loop terminating...
+[2024-07-24 18:07:10,746][05151] Stopping RolloutWorker_w2...
+[2024-07-24 18:07:10,746][03928] Component RolloutWorker_w2 stopped!
+[2024-07-24 18:07:10,749][05151] Loop rollout_proc2_evt_loop terminating...
+[2024-07-24 18:07:10,760][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000793_3248128.pth
+[2024-07-24 18:07:10,779][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2024-07-24 18:07:10,957][05135] Stopping LearnerWorker_p0...
+[2024-07-24 18:07:10,957][05135] Loop learner_proc0_evt_loop terminating...
+[2024-07-24 18:07:10,956][03928] Component LearnerWorker_p0 stopped!
+[2024-07-24 18:07:10,960][03928] Waiting for process learner_proc0 to stop...
+[2024-07-24 18:07:12,362][03928] Waiting for process inference_proc0-0 to join...
+[2024-07-24 18:07:12,365][03928] Waiting for process rollout_proc0 to join...
+[2024-07-24 18:07:14,367][03928] Waiting for process rollout_proc1 to join...
+[2024-07-24 18:07:14,370][03928] Waiting for process rollout_proc2 to join...
+[2024-07-24 18:07:14,376][03928] Waiting for process rollout_proc3 to join...
+[2024-07-24 18:07:14,379][03928] Waiting for process rollout_proc4 to join...
+[2024-07-24 18:07:14,383][03928] Waiting for process rollout_proc5 to join...
+[2024-07-24 18:07:14,388][03928] Waiting for process rollout_proc6 to join...
+[2024-07-24 18:07:14,391][03928] Waiting for process rollout_proc7 to join...
+[2024-07-24 18:07:14,394][03928] Batcher 0 profile tree view:
+batching: 28.4688, releasing_batches: 0.0284
+[2024-07-24 18:07:14,398][03928] InferenceWorker_p0-w0 profile tree view:
+wait_policy: 0.0000
+  wait_policy_total: 462.6672
+update_model: 9.9923
+  weight_update: 0.0024
+one_step: 0.0127
+  handle_policy_step: 634.6919
+    deserialize: 16.4988, stack: 3.3864, obs_to_device_normalize: 128.3574, forward: 338.0509, send_messages: 30.3013
+    prepare_outputs: 85.7805
+      to_cpu: 49.2792
+[2024-07-24 18:07:14,400][03928] Learner 0 profile tree view:
+misc: 0.0051, prepare_batch: 14.2455
+train: 75.5714
+  epoch_init: 0.0145, minibatch_init: 0.0154, losses_postprocess: 0.6793, kl_divergence: 0.8118, after_optimizer: 33.8778
+  calculate_losses: 28.0188
+    losses_init: 0.0038, forward_head: 1.4569, bptt_initial: 18.6248, tail: 1.2256, advantages_returns: 0.2762, losses: 3.9658
+    bptt: 2.1481
+      bptt_forward_core: 2.0234
+  update: 11.4792
+    clip: 1.0309
+[2024-07-24 18:07:14,401][03928] RolloutWorker_w0 profile tree view:
+wait_for_trajectories: 0.2901, enqueue_policy_requests: 123.5031, env_step: 889.5694, overhead: 16.4671, complete_rollouts: 7.3169
+save_policy_outputs: 21.4189
+  split_output_tensors: 8.8444
+[2024-07-24 18:07:14,402][03928] RolloutWorker_w7 profile tree view:
+wait_for_trajectories: 0.4289, enqueue_policy_requests: 126.3101, env_step: 889.9336, overhead: 16.4210, complete_rollouts: 7.5382
+save_policy_outputs: 21.6390
+  split_output_tensors: 8.7660
+[2024-07-24 18:07:14,404][03928] Loop Runner_EvtLoop terminating...
+[2024-07-24 18:07:14,406][03928] Runner profile tree view:
+main_loop: 1178.6859
+[2024-07-24 18:07:14,407][03928] Collected {0: 4005888}, FPS: 3398.6
+[2024-07-24 18:29:36,413][03928] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
+[2024-07-24 18:29:36,414][03928] Overriding arg 'num_workers' with value 1 passed from command line
+[2024-07-24 18:29:36,416][03928] Adding new argument 'no_render'=True that is not in the saved config file!
+[2024-07-24 18:29:36,418][03928] Adding new argument 'save_video'=True that is not in the saved config file!
+[2024-07-24 18:29:36,420][03928] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
+[2024-07-24 18:29:36,422][03928] Adding new argument 'video_name'=None that is not in the saved config file!
+[2024-07-24 18:29:36,424][03928] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
+[2024-07-24 18:29:36,426][03928] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
+[2024-07-24 18:29:36,429][03928] Adding new argument 'push_to_hub'=True that is not in the saved config file!
+[2024-07-24 18:29:36,430][03928] Adding new argument 'hf_repository'='dergky1/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
+[2024-07-24 18:29:36,432][03928] Adding new argument 'policy_index'=0 that is not in the saved config file!
+[2024-07-24 18:29:36,433][03928] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
+[2024-07-24 18:29:36,434][03928] Adding new argument 'train_script'=None that is not in the saved config file!
+[2024-07-24 18:29:36,435][03928] Adding new argument 'enjoy_script'=None that is not in the saved config file!
+[2024-07-24 18:29:36,436][03928] Using frameskip 1 and render_action_repeat=4 for evaluation
+[2024-07-24 18:29:36,471][03928] Doom resolution: 160x120, resize resolution: (128, 72)
+[2024-07-24 18:29:36,474][03928] RunningMeanStd input shape: (3, 72, 128)
+[2024-07-24 18:29:36,477][03928] RunningMeanStd input shape: (1,)
+[2024-07-24 18:29:36,493][03928] ConvEncoder: input_channels=3
+[2024-07-24 18:29:36,607][03928] Conv encoder output size: 512
+[2024-07-24 18:29:36,610][03928] Policy head output size: 512
+[2024-07-24 18:29:36,781][03928] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2024-07-24 18:29:37,578][03928] Num frames 100...
+[2024-07-24 18:29:37,723][03928] Num frames 200...
+[2024-07-24 18:29:37,856][03928] Num frames 300...
+[2024-07-24 18:29:37,991][03928] Num frames 400...
+[2024-07-24 18:29:38,132][03928] Num frames 500...
+[2024-07-24 18:29:38,261][03928] Num frames 600...
+[2024-07-24 18:29:38,388][03928] Num frames 700...
+[2024-07-24 18:29:38,518][03928] Num frames 800...
+[2024-07-24 18:29:38,661][03928] Avg episode rewards: #0: 16.670, true rewards: #0: 8.670
+[2024-07-24 18:29:38,663][03928] Avg episode reward: 16.670, avg true_objective: 8.670
+[2024-07-24 18:29:38,710][03928] Num frames 900...
+[2024-07-24 18:29:38,836][03928] Num frames 1000...
+[2024-07-24 18:29:38,963][03928] Num frames 1100...
+[2024-07-24 18:29:39,092][03928] Num frames 1200...
+[2024-07-24 18:29:39,232][03928] Num frames 1300...
+[2024-07-24 18:29:39,364][03928] Num frames 1400...
+[2024-07-24 18:29:39,501][03928] Avg episode rewards: #0: 13.810, true rewards: #0: 7.310
+[2024-07-24 18:29:39,504][03928] Avg episode reward: 13.810, avg true_objective: 7.310
+[2024-07-24 18:29:39,584][03928] Num frames 1500...
+[2024-07-24 18:29:39,781][03928] Num frames 1600...
+[2024-07-24 18:29:39,967][03928] Num frames 1700...
+[2024-07-24 18:29:40,158][03928] Num frames 1800...
+[2024-07-24 18:29:40,348][03928] Num frames 1900...
+[2024-07-24 18:29:40,526][03928] Num frames 2000...
+[2024-07-24 18:29:40,702][03928] Num frames 2100...
+[2024-07-24 18:29:40,898][03928] Num frames 2200...
+[2024-07-24 18:29:41,093][03928] Num frames 2300...
+[2024-07-24 18:29:41,330][03928] Avg episode rewards: #0: 15.307, true rewards: #0: 7.973
+[2024-07-24 18:29:41,333][03928] Avg episode reward: 15.307, avg true_objective: 7.973
+[2024-07-24 18:29:41,356][03928] Num frames 2400...
+[2024-07-24 18:29:41,554][03928] Num frames 2500...
+[2024-07-24 18:29:41,746][03928] Num frames 2600...
+[2024-07-24 18:29:41,948][03928] Num frames 2700...
+[2024-07-24 18:29:42,145][03928] Num frames 2800...
+[2024-07-24 18:29:42,281][03928] Num frames 2900...
+[2024-07-24 18:29:42,386][03928] Avg episode rewards: #0: 13.340, true rewards: #0: 7.340
+[2024-07-24 18:29:42,388][03928] Avg episode reward: 13.340, avg true_objective: 7.340
+[2024-07-24 18:29:42,479][03928] Num frames 3000...
+[2024-07-24 18:29:42,617][03928] Num frames 3100...
+[2024-07-24 18:29:42,754][03928] Num frames 3200...
+[2024-07-24 18:29:42,903][03928] Num frames 3300...
+[2024-07-24 18:29:43,038][03928] Num frames 3400...
+[2024-07-24 18:29:43,177][03928] Num frames 3500...
+[2024-07-24 18:29:43,316][03928] Num frames 3600...
+[2024-07-24 18:29:43,385][03928] Avg episode rewards: #0: 13.016, true rewards: #0: 7.216
+[2024-07-24 18:29:43,387][03928] Avg episode reward: 13.016, avg true_objective: 7.216
+[2024-07-24 18:29:43,507][03928] Num frames 3700...
+[2024-07-24 18:29:43,635][03928] Num frames 3800...
+[2024-07-24 18:29:43,764][03928] Num frames 3900...
+[2024-07-24 18:29:43,901][03928] Num frames 4000...
+[2024-07-24 18:29:44,029][03928] Num frames 4100...
+[2024-07-24 18:29:44,201][03928] Avg episode rewards: #0: 12.472, true rewards: #0: 6.972
+[2024-07-24 18:29:44,203][03928] Avg episode reward: 12.472, avg true_objective: 6.972
+[2024-07-24 18:29:44,227][03928] Num frames 4200...
+[2024-07-24 18:29:44,358][03928] Num frames 4300...
+[2024-07-24 18:29:44,491][03928] Num frames 4400...
+[2024-07-24 18:29:44,621][03928] Num frames 4500...
+[2024-07-24 18:29:44,754][03928] Num frames 4600...
+[2024-07-24 18:29:44,898][03928] Num frames 4700...
+[2024-07-24 18:29:45,031][03928] Num frames 4800...
+[2024-07-24 18:29:45,173][03928] Num frames 4900...
+[2024-07-24 18:29:45,304][03928] Num frames 5000...
+[2024-07-24 18:29:45,436][03928] Num frames 5100...
+[2024-07-24 18:29:45,548][03928] Avg episode rewards: #0: 13.776, true rewards: #0: 7.347
+[2024-07-24 18:29:45,549][03928] Avg episode reward: 13.776, avg true_objective: 7.347
+[2024-07-24 18:29:45,629][03928] Num frames 5200...
+[2024-07-24 18:29:45,776][03928] Num frames 5300...
+[2024-07-24 18:29:45,935][03928] Num frames 5400...
+[2024-07-24 18:29:46,086][03928] Num frames 5500...
+[2024-07-24 18:29:46,226][03928] Num frames 5600...
+[2024-07-24 18:29:46,362][03928] Num frames 5700...
+[2024-07-24 18:29:46,497][03928] Num frames 5800...
+[2024-07-24 18:29:46,627][03928] Num frames 5900...
+[2024-07-24 18:29:46,757][03928] Num frames 6000...
+[2024-07-24 18:29:46,887][03928] Num frames 6100...
+[2024-07-24 18:29:47,023][03928] Num frames 6200...
+[2024-07-24 18:29:47,135][03928] Avg episode rewards: #0: 14.930, true rewards: #0: 7.805
+[2024-07-24 18:29:47,137][03928] Avg episode reward: 14.930, avg true_objective: 7.805
+[2024-07-24 18:29:47,220][03928] Num frames 6300...
+[2024-07-24 18:29:47,349][03928] Num frames 6400...
+[2024-07-24 18:29:47,482][03928] Num frames 6500...
+[2024-07-24 18:29:47,612][03928] Num frames 6600...
+[2024-07-24 18:29:47,793][03928] Avg episode rewards: #0: 14.213, true rewards: #0: 7.436
+[2024-07-24 18:29:47,795][03928] Avg episode reward: 14.213, avg true_objective: 7.436
+[2024-07-24 18:29:47,809][03928] Num frames 6700...
+[2024-07-24 18:29:47,941][03928] Num frames 6800...
+[2024-07-24 18:29:48,084][03928] Num frames 6900...
+[2024-07-24 18:29:48,228][03928] Num frames 7000...
+[2024-07-24 18:29:48,363][03928] Num frames 7100...
+[2024-07-24 18:29:48,491][03928] Num frames 7200...
+[2024-07-24 18:29:48,627][03928] Num frames 7300...
+[2024-07-24 18:29:48,759][03928] Num frames 7400...
+[2024-07-24 18:29:48,898][03928] Num frames 7500...
+[2024-07-24 18:29:49,041][03928] Num frames 7600...
+[2024-07-24 18:29:49,176][03928] Num frames 7700...
+[2024-07-24 18:29:49,312][03928] Num frames 7800...
+[2024-07-24 18:29:49,452][03928] Num frames 7900...
+[2024-07-24 18:29:49,585][03928] Num frames 8000...
+[2024-07-24 18:29:49,715][03928] Num frames 8100...
+[2024-07-24 18:29:49,851][03928] Num frames 8200...
+[2024-07-24 18:29:49,982][03928] Num frames 8300...
+[2024-07-24 18:29:50,081][03928] Avg episode rewards: #0: 16.924, true rewards: #0: 8.324
+[2024-07-24 18:29:50,082][03928] Avg episode reward: 16.924, avg true_objective: 8.324
+[2024-07-24 18:30:43,566][03928] Replay video saved to /content/train_dir/default_experiment/replay.mp4!