Phi-3.5-Family
Collection
Quantifying and transforming models from Phi-3.5 Family
•
8 items
•
Updated
Phi-3.5-mini-instruct-onnx-cpu Unofficial version
Note: This is unoffical version,just for test and dev.
This is a Phi-3.5-mini-instruct version of ONNX CPU, based on ONNX Runtime for GenAI https://github.com/microsoft/onnxruntime-genai. Convert with the following command
1. Install the SDK
pip install torch transformers onnx onnxruntime
pip install --pre onnxruntime-genai
2. Convert CPU ONNX Support
python3 -m onnxruntime_genai.models.builder -m microsoft/Phi-3.5-mini-instruct -o ./onnx-cpu -p int4 -e cpu -c ./Phi-3.5-mini-instruct
This is a conversion, but no specific optimization has been done. Please look forward to the official version.
- Downloads last month
- 11
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.