# alfworld_workflow_preference_dpo_1.0 This model is a fine-tuned version of [LLaMA-Factory/saves/llama3.1-8b/alfworld_workflow](https://huggingface.co/LLaMA-Factory/saves/llama3.1-8b/alfworld_workflow) on the alfworld_workflow_preference_1 dataset. It achieves the following results on the evaluation set: - Loss: 0.8390 - Rewards/chosen: -0.5836 - Rewards/rejected: -1.2646 - Rewards/accuracies: 0.6318 - Rewards/margins: 0.6810 - Logps/chosen: -12.9009 - Logps/rejected: -19.8890 - Logits/chosen: -0.3349 - Logits/rejected: -0.3405 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 2 - eval_batch_size: 1 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - gradient_accumulation_steps: 4 - total_train_batch_size: 32 - total_eval_batch_size: 4 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.03 - num_epochs: 3.0 ### Training results ### Framework versions - Transformers 4.46.1 - Pytorch 2.5.1+cu124 - Datasets 3.1.0 - Tokenizers 0.20.3