runtime error

Exit code: 1. Reason: DME.md: 0%| | 0.00/9.66k [00:00<?, ?B/s] README.md: 100%|██████████| 9.66k/9.66k [00:00<00:00, 48.5MB/s] exp-schp-201908261155-lip.pth: 0%| | 0.00/267M [00:00<?, ?B/s] exp-schp-201908261155-lip.pth: 100%|█████████▉| 267M/267M [00:00<00:00, 272MB/s] exp-schp-201908301523-atr.pth: 0%| | 0.00/267M [00:00<?, ?B/s] exp-schp-201908301523-atr.pth: 100%|█████████▉| 267M/267M [00:00<00:00, 286MB/s] model.safetensors: 0%| | 0.00/198M [00:00<?, ?B/s] model.safetensors: 100%|█████████▉| 198M/198M [00:00<00:00, 249MB/s] pytorch_lora_weights.safetensors: 0%| | 0.00/37.4M [00:00<?, ?B/s] pytorch_lora_weights.safetensors: 100%|██████████| 37.4M/37.4M [00:00<00:00, 40.1MB/s] model.safetensors: 0%| | 0.00/198M [00:00<?, ?B/s] model.safetensors: 100%|█████████▉| 198M/198M [00:01<00:00, 154MB/s] model.safetensors: 100%|█████████▉| 198M/198M [00:01<00:00, 153MB/s] model.safetensors: 0%| | 0.00/198M [00:00<?, ?B/s] model.safetensors: 100%|█████████▉| 198M/198M [00:00<00:00, 253MB/s] /usr/local/lib/python3.10/site-packages/torch/cuda/memory.py:343: FutureWarning: torch.cuda.reset_max_memory_allocated now calls torch.cuda.reset_peak_memory_stats, which resets /all/ peak memory stats. warnings.warn( Traceback (most recent call last): File "/home/user/app/app.py", line 128, in <module> flush() File "/home/user/app/app.py", line 125, in flush torch.cuda.reset_max_memory_allocated() File "/usr/local/lib/python3.10/site-packages/torch/cuda/memory.py", line 348, in reset_max_memory_allocated return reset_peak_memory_stats(device=device) File "/usr/local/lib/python3.10/site-packages/torch/cuda/memory.py", line 322, in reset_peak_memory_stats return torch._C._cuda_resetPeakMemoryStats(device) RuntimeError: invalid argument to reset_peak_memory_stats

Container logs: