metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:3284
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-m-v1.5
widget:
- source_sentence: >-
How do I register and activate a stack with a new orchestrator using
ZenML?
sentences:
- >-
hestrator registered and part of our active stack:zenml orchestrator
register <ORCHESTRATOR_NAME> \
--flavor=airflow \
--local=True # set this to `False` if using a remote Airflow deployment
zenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME> ... --set
Due to dependency conflicts, we need to install the Python packages to
start a local Airflow server in a separate Python environment.
server dependencies
python -m venv airflow_server_environment
source airflow_server_environment/bin/activate
pip install "apache-airflow==2.4.0"
"apache-airflow-providers-docker<3.8.0" "pydantic~=2.7.1"
Before starting the local Airflow server, we can set a few environment
variables to configure it:
AIRFLOW_HOME: This variable defines the location where the Airflow
server stores its database and configuration files. The default value is
~/airflow.
AIRFLOW__CORE__DAGS_FOLDER: This variable defines the location where the
Airflow server looks for DAG files. The default value is
<AIRFLOW_HOME>/dags.
AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL: This variable controls how
often the Airflow scheduler checks for new or updated DAGs. By default,
the scheduler will check for new DAGs every 30 seconds. This variable
can be used to increase or decrease the frequency of the checks,
depending on the specific needs of your pipeline.
When running this on MacOS, you might need to set the no_proxy
environment variable to prevent crashes due to a bug in Airflow (see
this page for more information):
export no_proxy=*
We can now start the local Airflow server by running the following
command:
running this command
airflow standalone
- >-
ta stores you want to migrate, then upgrade ZenML.Decide the ZenML
deployment model that you want to follow for your projects. See the
ZenML deployment documentation for available deployment scenarios. If
you decide on using a local or remote ZenML server to manage your
pipelines, make sure that you first connect your client to it by running
zenml connect.
Use the zenml pipeline runs migrate CLI command to migrate your old
pipeline runs:
If you want to migrate from a local SQLite metadata store, you only need
to pass the path to the metadata store to the command, e.g.:
zenml pipeline runs migrate PATH/TO/LOCAL/STORE/metadata.db
If you would like to migrate any other store, you will need to set
--database_type=mysql and provide the MySQL host, username, and password
in addition to the database, e.g.:
zenml pipeline runs migrate DATABASE_NAME \
--database_type=mysql \
--mysql_host=URL/TO/MYSQL \
--mysql_username=MYSQL_USERNAME \
--mysql_password=MYSQL_PASSWORD
💾 The New Way (CLI Command Cheat Sheet)
Deploy the server
zenml deploy --aws (maybe don’t do this :) since it spins up
infrastructure on AWS…)
Spin up a local ZenML Server
zenml up
Connect to a pre-existing server
zenml connect (pass in URL / etc, or zenml connect --config + yaml file)
List your deployed server details
zenml status
The ZenML Dashboard is now available
The new ZenML Dashboard is now bundled into the ZenML Python package and
can be launched directly from Python. The source code lives in the ZenML
Dashboard repository.
To launch it locally, simply run zenml up on your machine and follow the
instructions:
$ zenml up
Deploying a local ZenML server with name 'local'.
Connecting ZenML to the 'local' local ZenML server
(http://127.0.0.1:8237).
Updated the global store configuration.
Connected ZenML to the 'local' local ZenML server
(http://127.0.0.1:8237).
The local ZenML dashboard is available at 'http://127.0.0.1:8237'. You
can
connect to it using the 'default' username and an empty password.
- |-
🐍Configure Python environments
Navigating multiple development environments.
PreviousHyperAI Service ConnectorNextHandling dependencies
Last updated 21 days ago
- source_sentence: >-
How do you build a simple machine learning pipeline using ZenML decorators
in the code?
sentences:
- >-
Develop a custom data validator
How to develop a custom data validator
Before diving into the specifics of this component type, it is
beneficial to familiarize yourself with our general guide to writing
custom component flavors in ZenML. This guide provides an essential
understanding of ZenML's component flavor concepts.
Base abstraction in progress!
We are actively working on the base abstraction for the Data Validators,
which will be available soon. As a result, their extension is not
recommended at the moment. When you are selecting a data validator for
your stack, you can use one of the existing flavors.
If you need to implement your own Data Validator flavor, you can still
do so, but keep in mind that you may have to refactor it when the base
abstraction is updated.
ZenML comes equipped with Data Validator implementations that integrate
a variety of data logging and validation libraries, frameworks and
platforms. However, if you need to use a different library or service as
a backend for your ZenML Data Validator, you can extend ZenML to provide
your own custom Data Validator implementation.
Build your own custom data validator
If you want to implement your own custom Data Validator, you can follow
the following steps:
Create a class which inherits from the BaseDataValidator class and
override one or more of the abstract methods, depending on the
capabilities of the underlying library/service that you want to
integrate.
If you need any configuration, you can create a class which inherits
from the BaseDataValidatorConfig class.
Bring both of these classes together by inheriting from the
BaseDataValidatorFlavor.
(Optional) You should also provide some standard steps that others can
easily insert into their pipelines for instant access to data validation
features.
Once you are done with the implementation, you can register it through
the CLI. Please ensure you point to the flavor class via dot notation:
- >2-
This is us if you want to put faces to the names!However, in order to improve ZenML and understand how it is being used, we need to use analytics to have an overview of how it is used 'in the wild'. This not only helps us find bugs but also helps us prioritize features and commands that might be useful in future releases. If we did not have this information, all we really get is pip download statistics and chatting with people directly, which while being valuable, is not enough to seriously better the tool as a whole.
How does ZenML collect these statistics?
We use Segment as the data aggregation library for all our analytics.
However, before any events get sent to Segment, they first go through a
central ZenML analytics server. This added layer allows us to put
various countermeasures to incidents such as getting spammed with events
and enables us to have a more optimized tracking process.
The client code is entirely visible and can be seen in the analytics
module of our main repository.
If I share my email, will you spam me?
No, we won't. Our sole purpose of contacting you will be to ask for
feedback (e.g. in the shape of a user interview). These interviews help
the core team understand usage better and prioritize feature requests.
If you have any concerns about data privacy and the usage of personal
information, please contact us, and we will try to alleviate any
concerns as soon as possible.
Version mismatch (downgrading)
If you've recently downgraded your ZenML version to an earlier release
or installed a newer version on a different environment on the same
machine, you might encounter an error message when running ZenML that
says:
`The ZenML global configuration version (%s) is higher than the version
of ZenML
currently being used (%s).`
We generally recommend using the latest ZenML version. However, there
might be cases where you need to match the global configuration version
with the version of ZenML installed in the current environment. To do
this, run the following command:
zenml downgrade
- >-
⛓️Build a pipeline
Building pipelines is as simple as adding the `@step` and `@pipeline`
decorators to your code.
@step
def load_data() -> dict:
training_data = [[1, 2], [3, 4], [5, 6]]
labels = [0, 1, 0]
return {'features': training_data, 'labels': labels}
@step
def train_model(data: dict) -> None:
total_features = sum(map(sum, data['features']))
total_labels = sum(data['labels'])
print(f"Trained model using {len(data['features'])} data points. "
f"Feature sum is {total_features}, label sum is {total_labels}")
@pipeline # This function combines steps together
def simple_ml_pipeline():
dataset = load_data()
train_model(dataset)
You can now run this pipeline by simply calling the function:
simple_ml_pipeline()
When this pipeline is executed, the run of the pipeline gets logged to
the ZenML dashboard where you can now go to look at its DAG and all the
associated metadata. To access the dashboard you need to have a ZenML
server either running locally or remotely. See our documentation on this
here.
Check below for more advanced ways to build and interact with your
pipeline.
Configure pipeline/step parameters
Name and annotate step outputs
Control caching behavior
Run pipeline from a pipeline
Control the execution order of steps
Customize the step invocation ids
Name your pipeline runs
Use failure/success hooks
Hyperparameter tuning
Attach metadata to steps
Fetch metadata within steps
Fetch metadata during pipeline composition
Enable or disable logs storing
Special Metadata Types
Access secrets in a step
PreviousBest practicesNextUse pipeline/step parameters
Last updated 1 month ago
- source_sentence: >-
How can I integrate Large Language Models (LLMs) into my MLOps workflows
using ZenML?
sentences:
- >-
🦜LLMOps guide
Leverage the power of LLMs in your MLOps workflows with ZenML.
Welcome to the ZenML LLMOps Guide, where we dive into the exciting world
of Large Language Models (LLMs) and how to integrate them seamlessly
into your MLOps pipelines using ZenML. This guide is designed for ML
practitioners and MLOps engineers looking to harness the potential of
LLMs while maintaining the robustness and scalability of their
workflows.
In this guide, we'll explore various aspects of working with LLMs in
ZenML, including:
RAG with ZenML
RAG in 85 lines of code
Understanding Retrieval-Augmented Generation (RAG)
Data ingestion and preprocessing
Embeddings generation
Storing embeddings in a vector database
Basic RAG inference pipeline
Evaluation and metrics
Evaluation in 65 lines of code
Retrieval evaluation
Generation evaluation
Evaluation in practice
Reranking for better retrieval
Understanding reranking
Implementing reranking in ZenML
Evaluating reranking performance
Improve retrieval by finetuning embeddings
Synthetic data generation
Finetuning embeddings with Sentence Transformers
Evaluating finetuned embeddings
Finetuning LLMs with ZenML
To follow along with the examples and tutorials in this guide, ensure
you have a Python environment set up with ZenML installed. Familiarity
with the concepts covered in the Starter Guide and Production Guide is
recommended.
We'll showcase a specific application over the course of this LLM guide,
showing how you can work from a simple RAG pipeline to a more complex
setup that involves finetuning embeddings, reranking retrieved
documents, and even finetuning the LLM itself. We'll do this all for a
use case relevant to ZenML: a question answering system that can provide
answers to common questions about ZenML. This will help you understand
how to apply the concepts covered in this guide to your own projects.
- |2-
data with tags
Get arbitrary artifacts in a stepHandle custom data types
Load artifacts into memory
Datasets in ZenML
Manage big data
Skipping materialization
Passing artifacts between pipelines
Register Existing Data as a ZenML Artifact
📊Visualizing artifacts
Default visualizations
Creating custom visualizations
Displaying visualizations in the dashboard
Disabling visualizations
🪆Use the Model Control Plane
Registering a Model
Deleting a Model
Associate a pipeline with a Model
Connecting artifacts via a Model
Controlling Model versions
Load a Model in code
Promote a Model
Linking model binaries/data to a Model
Load artifacts from Model
📈Track metrics and metadata
Attach metadata to a model
Attach metadata to an artifact
Attach metadata to steps
Group metadata
Special Metadata Types
Fetch metadata within steps
Fetch metadata during pipeline composition
👨🎤Popular integrations
Run on AWS
Run on GCP
Run on Azure
Kubeflow
Kubernetes
MLflow
Skypilot
🔌Connect services (AWS, GCP, Azure, K8s etc)
Service Connectors guide
Security best practices
Docker Service Connector
Kubernetes Service Connector
AWS Service Connector
GCP Service Connector
Azure Service Connector
HyperAI Service Connector
🐍Configure Python environments
Handling dependencies
Configure the server environment
🔌Connect to a server
Connect in with your User (interactive)
Connect with a Service Account
🔐Interact with secrets
🐞Debug and solve issues
🤝Contribute to ZenML
Implement a custom integration
Stack Components
📜Overview
🔋Orchestrators
Local Orchestrator
Local Docker Orchestrator
Kubeflow Orchestrator
Kubernetes Orchestrator
Google Cloud VertexAI Orchestrator
AWS Sagemaker Orchestrator
AzureML Orchestrator
Databricks Orchestrator
Tekton Orchestrator
Airflow Orchestrator
Skypilot VM Orchestrator
HyperAI Orchestrator
Lightning AI Orchestrator
Develop a custom orchestrator
🏪Artifact Stores
Local Artifact Store
Amazon Simple Cloud Storage (S3)
Google Cloud Storage (GCS)
- >-
Troubleshoot the deployed server
Troubleshooting tips for your ZenML deployment
In this document, we will go over some common issues that you might face
when deploying ZenML and how to solve them.
Viewing logs
Analyzing logs is a great way to debug issues. Depending on whether you
have a Kubernetes (using Helm or zenml deploy) or a Docker deployment,
you can view the logs in different ways.
If you are using Kubernetes, you can view the logs of the ZenML server
using the following method:
Check all pods that are running your ZenML deployment.
kubectl -n <KUBERNETES_NAMESPACE> get pods
If you see that the pods aren't running, you can use the command below
to get the logs for all pods at once.
kubectl -n <KUBERNETES_NAMESPACE> logs -l app.kubernetes.io/name=zenml
Note that the error can either be from the zenml-db-init container that
connects to the MySQL database or from the zenml container that runs the
server code. If the get pods command shows that the pod is failing in
the Init state then use zenml-db-init as the container name, otherwise
use zenml.
kubectl -n <KUBERNETES_NAMESPACE> logs -l app.kubernetes.io/name=zenml
-c <CONTAINER_NAME>
You can also use the --tail flag to limit the number of lines to show or
the --follow flag to follow the logs in real-time.
If you are using Docker, you can view the logs of the ZenML server using
the following method:
If you used the zenml up --docker CLI command to deploy the Docker ZenML
server, you can check the logs with the command:
zenml logs -f
If you used the docker run command to manually deploy the Docker ZenML
server, you can check the logs with the command:
docker logs zenml -f
If you used the docker compose command to manually deploy the Docker
ZenML server, you can check the logs with the command:
docker compose -p zenml logs -f
Fixing database connection problems
- source_sentence: How can you disable artifact visualization in ZenML?
sentences:
- |-
Secret management
Configuring the secrets store.
PreviousCustom secret storesNextZenML Pro
Last updated 21 days ago
- >2-
visit our PyPi package page.
Running with Dockerzenml is also available as a Docker image hosted
publicly on DockerHub. Use the following command to get started in a
bash environment with zenml available:
docker run -it zenmldocker/zenml /bin/bash
If you would like to run the ZenML server with Docker:
docker run -it -d -p 8080:8080 zenmldocker/zenml-server
Deploying the server
Though ZenML can run entirely as a pip package on a local system,
complete with the dashboard. You can do this easily:
pip install "zenml[server]"
zenml up
However, advanced ZenML features are dependent on a centrally-deployed
ZenML server accessible to other MLOps stack components. You can read
more about it here.
For the deployment of ZenML, you have the option to either self-host it
or register for a free ZenML Pro account.
PreviousIntroductionNextCore concepts
Last updated 20 days ago
- >-
Disabling visualizations
Disabling visualizations.
If you would like to disable artifact visualization altogether, you can
set enable_artifact_visualization at either pipeline or step level:
@step(enable_artifact_visualization=False)
def my_step():
...
@pipeline(enable_artifact_visualization=False)
def my_pipeline():
...
PreviousDisplaying visualizations in the dashboardNextUse the Model
Control Plane
Last updated 21 days ago
- source_sentence: >-
How can I programmatically manage secrets using the ZenML Client API, and
what are some of the methods available for tasks like fetching, updating,
and deleting secrets?
sentences:
- >-
tack:
zenml stack register-secrets [<STACK_NAME>]The ZenML client API offers a
programmatic interface to create, e.g.:
from zenml.client import Client
client = Client()
client.create_secret(
name="my_secret",
values={
"username": "admin",
"password": "abc123"
}
)
Other Client methods used for secrets management include get_secret to
fetch a secret by name or id, update_secret to update an existing
secret, list_secrets to query the secrets store using a variety of
filtering and sorting criteria, and delete_secret to delete a secret.
The full Client API reference is available here.
Set scope for secrets
ZenML secrets can be scoped to a user. This allows you to create secrets
that are only accessible to one user.
By default, all created secrets are scoped to the active user. To create
a secret and scope it to your active user instead, you can pass the
--scope argument to the CLI command:
zenml secret create <SECRET_NAME> \
--scope user \
--<KEY_1>=<VALUE_1> \
--<KEY_2>=<VALUE_2>
Scopes also act as individual namespaces. When you are referencing a
secret by name in your pipelines and stacks, ZenML will look for a
secret with that name scoped to the active user.
Accessing registered secrets
Reference secrets in stack component attributes and settings
Some of the components in your stack require you to configure them with
sensitive information like passwords or tokens, so they can connect to
the underlying infrastructure. Secret references allow you to configure
these components in a secure way by not specifying the value directly
but instead referencing a secret by providing the secret name and key.
Referencing a secret for the value of any string attribute of your stack
components, simply specify the attribute using the following syntax:
{{<SECRET_NAME>.<SECRET_KEY>}}
For example:
- >-
y to the active stack
zenml stack update -c <NAME>Additionally, we'll need to log in to the
container registry so Docker can pull and push images. This will require
your DockerHub account name and either your password or preferably a
personal access token.
docker login
For more information and a full list of configurable attributes of the
dockerhub container registry, check out the SDK Docs .
PreviousDefault Container RegistryNextAmazon Elastic Container Registry
(ECR)
Last updated 4 months ago
- >-
nect the stack component to the Service Connector:$ zenml step-operator
register <NAME> --flavor kubernetes
Running with active stack: 'default' (repository)
Successfully registered step operator `<NAME>`.
$ zenml service-connector list-resources --resource-type
kubernetes-cluster -e
The following 'kubernetes-cluster' resources can be accessed by service
connectors that you have configured:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │
CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu │ 🔶
aws │ 🌀 kubernetes-cluster │ kubeflowmultitenant ┃
┃ │
│ │ │ zenbox ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us │ 🔶
aws │ 🌀 kubernetes-cluster │ zenhacks-cluster ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi │ 🔵
gcp │ 🌀 kubernetes-cluster │ zenml-test-cluster ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: zenml/finetuned-snowflake-arctic-embed-m-v1.5
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 0.1917808219178082
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5095890410958904
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6986301369863014
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.810958904109589
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.1917808219178082
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.16986301369863013
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13972602739726026
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08109589041095888
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.1917808219178082
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5095890410958904
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6986301369863014
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.810958904109589
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.490826354124735
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.38868232224396587
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.3947220516402755
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.19726027397260273
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5150684931506849
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6931506849315069
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8191780821917808
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.19726027397260273
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.17168949771689496
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13863013698630136
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08191780821917807
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.19726027397260273
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5150684931506849
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6931506849315069
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8191780821917808
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.4973578695114294
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.3948053924766253
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.39989299235069015
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.19452054794520549
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5013698630136987
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.673972602739726
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7835616438356164
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.19452054794520549
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.16712328767123283
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1347945205479452
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07835616438356163
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.19452054794520549
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5013698630136987
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.673972602739726
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7835616438356164
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.47814279525126957
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.38079147640791483
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.38821398163789955
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.1780821917808219
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.4602739726027397
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6547945205479452
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7753424657534247
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.1780821917808219
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.15342465753424658
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13095890410958905
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07753424657534246
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.1780821917808219
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4602739726027397
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6547945205479452
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7753424657534247
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.4625139710379368
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.36313002826701446
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.37046958498969434
name: Cosine Map@100
zenml/finetuned-snowflake-arctic-embed-m-v1.5
This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: Snowflake/snowflake-arctic-embed-m-v1.5
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5")
sentences = [
'How can I programmatically manage secrets using the ZenML Client API, and what are some of the methods available for tasks like fetching, updating, and deleting secrets?',
'tack:\n\nzenml stack register-secrets [<STACK_NAME>]The ZenML client API offers a programmatic interface to create, e.g.:\n\nfrom zenml.client import Client\n\nclient = Client()\nclient.create_secret(\n name="my_secret",\n values={\n "username": "admin",\n "password": "abc123"\n }\n)\n\nOther Client methods used for secrets management include get_secret to fetch a secret by name or id, update_secret to update an existing secret, list_secrets to query the secrets store using a variety of filtering and sorting criteria, and delete_secret to delete a secret. The full Client API reference is available here.\n\nSet scope for secrets\n\nZenML secrets can be scoped to a user. This allows you to create secrets that are only accessible to one user.\n\nBy default, all created secrets are scoped to the active user. To create a secret and scope it to your active user instead, you can pass the --scope argument to the CLI command:\n\nzenml secret create <SECRET_NAME> \\\n --scope user \\\n --<KEY_1>=<VALUE_1> \\\n --<KEY_2>=<VALUE_2>\n\nScopes also act as individual namespaces. When you are referencing a secret by name in your pipelines and stacks, ZenML will look for a secret with that name scoped to the active user.\n\nAccessing registered secrets\n\nReference secrets in stack component attributes and settings\n\nSome of the components in your stack require you to configure them with sensitive information like passwords or tokens, so they can connect to the underlying infrastructure. Secret references allow you to configure these components in a secure way by not specifying the value directly but instead referencing a secret by providing the secret name and key. Referencing a secret for the value of any string attribute of your stack components, simply specify the attribute using the following syntax: {{<SECRET_NAME>.<SECRET_KEY>}}\n\nFor example:\n\n# Register a secret called `mlflow_secret` with key-value pairs for the\n# username and password to authenticate with the MLflow tracking server',
"nect the stack component to the Service Connector:$ zenml step-operator register <NAME> --flavor kubernetes\nRunning with active stack: 'default' (repository)\nSuccessfully registered step operator `<NAME>`.\n\n$ zenml service-connector list-resources --resource-type kubernetes-cluster -e\nThe following 'kubernetes-cluster' resources can be accessed by service connectors that you have configured:\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓\n┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu │ 🔶 aws │ 🌀 kubernetes-cluster │ kubeflowmultitenant ┃\n┃ │ │ │ │ zenbox ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us │ 🔶 aws │ 🌀 kubernetes-cluster │ zenhacks-cluster ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi │ 🔵 gcp │ 🌀 kubernetes-cluster │ zenml-test-cluster ┃\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.1918 |
| cosine_accuracy@3 |
0.5096 |
| cosine_accuracy@5 |
0.6986 |
| cosine_accuracy@10 |
0.811 |
| cosine_precision@1 |
0.1918 |
| cosine_precision@3 |
0.1699 |
| cosine_precision@5 |
0.1397 |
| cosine_precision@10 |
0.0811 |
| cosine_recall@1 |
0.1918 |
| cosine_recall@3 |
0.5096 |
| cosine_recall@5 |
0.6986 |
| cosine_recall@10 |
0.811 |
| cosine_ndcg@10 |
0.4908 |
| cosine_mrr@10 |
0.3887 |
| cosine_map@100 |
0.3947 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.1973 |
| cosine_accuracy@3 |
0.5151 |
| cosine_accuracy@5 |
0.6932 |
| cosine_accuracy@10 |
0.8192 |
| cosine_precision@1 |
0.1973 |
| cosine_precision@3 |
0.1717 |
| cosine_precision@5 |
0.1386 |
| cosine_precision@10 |
0.0819 |
| cosine_recall@1 |
0.1973 |
| cosine_recall@3 |
0.5151 |
| cosine_recall@5 |
0.6932 |
| cosine_recall@10 |
0.8192 |
| cosine_ndcg@10 |
0.4974 |
| cosine_mrr@10 |
0.3948 |
| cosine_map@100 |
0.3999 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.1945 |
| cosine_accuracy@3 |
0.5014 |
| cosine_accuracy@5 |
0.674 |
| cosine_accuracy@10 |
0.7836 |
| cosine_precision@1 |
0.1945 |
| cosine_precision@3 |
0.1671 |
| cosine_precision@5 |
0.1348 |
| cosine_precision@10 |
0.0784 |
| cosine_recall@1 |
0.1945 |
| cosine_recall@3 |
0.5014 |
| cosine_recall@5 |
0.674 |
| cosine_recall@10 |
0.7836 |
| cosine_ndcg@10 |
0.4781 |
| cosine_mrr@10 |
0.3808 |
| cosine_map@100 |
0.3882 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.1781 |
| cosine_accuracy@3 |
0.4603 |
| cosine_accuracy@5 |
0.6548 |
| cosine_accuracy@10 |
0.7753 |
| cosine_precision@1 |
0.1781 |
| cosine_precision@3 |
0.1534 |
| cosine_precision@5 |
0.131 |
| cosine_precision@10 |
0.0775 |
| cosine_recall@1 |
0.1781 |
| cosine_recall@3 |
0.4603 |
| cosine_recall@5 |
0.6548 |
| cosine_recall@10 |
0.7753 |
| cosine_ndcg@10 |
0.4625 |
| cosine_mrr@10 |
0.3631 |
| cosine_map@100 |
0.3705 |
Training Details
Training Dataset
json
- Dataset: json
- Size: 3,284 training samples
- Columns:
positive and anchor
- Approximate statistics based on the first 1000 samples:
|
positive |
anchor |
| type |
string |
string |
| details |
- min: 11 tokens
- mean: 22.84 tokens
- max: 48 tokens
|
- min: 17 tokens
- mean: 322.65 tokens
- max: 512 tokens
|
- Samples:
| positive |
anchor |
How can impersonating accounts and assuming roles enhance flexibility and control in authentication methods for Service Connectors within ZenML? |
ethods.
Impersonating accounts and assuming rolesThese types of authentication methods require more work to set up because multiple permission-bearing accounts and roles need to be provisioned in advance depending on the target audience. On the other hand, they also provide the most flexibility and control. Despite their operational cost, if you are a platform engineer and have the infrastructure know-how necessary to understand and set up the authentication resources, this is for you.
These authentication methods deliver another way of configuring long-lived credentials in your Service Connectors without exposing them to clients. They are especially useful as an alternative to cloud provider Service Connectors authentication methods that do not support automatically downscoping the permissions of issued temporary tokens.
The processes of account impersonation and role assumption are very similar and can be summarized as follows:
you configure a Service Connector with long-lived credentials associated with a primary user account or primary service account (preferable). As a best practice, it is common to attach a reduced set of permissions or even no permissions to these credentials other than those that allow the account impersonation or role assumption operation. This makes it more difficult to do any damage if the primary credentials are accidentally leaked.
in addition to the primary account and its long-lived credentials, you also need to provision one or more secondary access entities in the cloud platform bearing the effective permissions that will be needed to access the target resource(s):
one or more IAM roles (to be assumed)
one or more service accounts (to be impersonated)
the Service Connector configuration also needs to contain the name of a target IAM role to be assumed or a service account to be impersonated. |
How can I specify the kubeflow_hostname and provide authentication details for multi-tenant Kubeflow deployments when using ZenML? |
strator_flavor import KubeflowOrchestratorSettingskubeflow_settings = KubeflowOrchestratorSettings(
client_args={},
user_namespace="my_namespace",
pod_settings={
"affinity": {...},
"tolerations": [...]
}
)
@pipeline(
settings={
"orchestrator": kubeflow_settings
}
)
This allows specifying client arguments, user namespace, pod affinity/tolerations, and more.
Multi-Tenancy Deployments
For multi-tenant Kubeflow deployments, specify the kubeflow_hostname ending in /pipeline when registering the orchestrator:
zenml orchestrator register <br> --flavor=kubeflow <br> --kubeflow_hostname= # e.g. https://mykubeflow.example.com/pipeline
And provide the namespace, username and password in the orchestrator settings:
kubeflow_settings = KubeflowOrchestratorSettings(
client_username="admin",
client_password="abc123",
user_namespace="namespace_name"
)
@pipeline(
settings={
"orchestrator": kubeflow_settings
}
)
For more advanced options and details, refer to the full Kubeflow Orchestrator documentation.
PreviousRun on AzureNextKubernetes
Last updated 21 days ago |
How can I contribute to the ZenML project? |
ding ZenML to learn more!
Do you support Windows?ZenML officially supports Windows if you're using WSL. Much of ZenML will also work on Windows outside a WSL environment, but we don't officially support it and some features don't work (notably anything that requires spinning up a server process).
Do you support Macs running on Apple Silicon?
Yes, ZenML does support Macs running on Apple Silicon. You just need to make sure that you set the following environment variable:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
This is a known issue with how forking works on Macs running on Apple Silicon and it will enable you to use ZenML and the server. This environment variable is needed if you are working with a local server on your Mac, but if you're just using ZenML as a client / CLI and connecting to a deployed server then you don't need to set it.
How can I make ZenML work with my custom tool? How can I extend or build on ZenML?
This depends on the tool and its respective MLOps category. We have a full guide on this over here!
How can I contribute?
We develop ZenML together with our community! To get involved, the best way to get started is to select any issue from the good-first-issue label. If you would like to contribute, please review our Contributing Guide for all relevant details.
How can I speak with the community?
The first point of the call should be our Slack group. Ask your questions about bugs or specific use cases and someone from the core team will respond.
Which license does ZenML use?
ZenML is distributed under the terms of the Apache License Version 2.0. A complete version of the license is available in the LICENSE.md in this repository. Any contribution made to this project will be licensed under the Apache License Version 2.0.
PreviousCommunity & content
Last updated 3 months ago |
- Loss:
MatryoshkaLoss with these parameters:{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
384,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1
],
"n_dims_per_step": -1
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: epochper_device_train_batch_size: 4per_device_eval_batch_size: 16gradient_accumulation_steps: 16learning_rate: 2e-05num_train_epochs: 4lr_scheduler_type: cosinewarmup_ratio: 0.1tf32: Falseload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 4per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 16eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 4max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Falsefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Falselocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseeval_use_gather_object: Falsebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch |
Step |
Training Loss |
dim_384_cosine_map@100 |
dim_256_cosine_map@100 |
dim_128_cosine_map@100 |
dim_64_cosine_map@100 |
| 0.1949 |
10 |
1.0787 |
- |
- |
- |
- |
| 0.3898 |
20 |
0.5131 |
- |
- |
- |
- |
| 0.5847 |
30 |
0.2011 |
- |
- |
- |
- |
| 0.7795 |
40 |
0.1264 |
- |
- |
- |
- |
| 0.9744 |
50 |
0.0754 |
- |
- |
- |
- |
| 0.9939 |
51 |
- |
0.3934 |
0.3870 |
0.3630 |
0.3397 |
| 1.1693 |
60 |
0.0581 |
- |
- |
- |
- |
| 1.3642 |
70 |
0.0543 |
- |
- |
- |
- |
| 1.5591 |
80 |
0.0467 |
- |
- |
- |
- |
| 1.7540 |
90 |
0.0154 |
- |
- |
- |
- |
| 1.9488 |
100 |
0.0407 |
- |
- |
- |
- |
| 1.9878 |
102 |
- |
0.4025 |
0.4052 |
0.387 |
0.3672 |
| 2.1437 |
110 |
0.0098 |
- |
- |
- |
- |
| 2.3386 |
120 |
0.0229 |
- |
- |
- |
- |
| 2.5335 |
130 |
0.0184 |
- |
- |
- |
- |
| 2.7284 |
140 |
0.009 |
- |
- |
- |
- |
| 2.9233 |
150 |
0.0308 |
- |
- |
- |
- |
| 2.9817 |
153 |
- |
0.3919 |
0.4006 |
0.3855 |
0.3718 |
| 3.1181 |
160 |
0.0048 |
- |
- |
- |
- |
| 3.3130 |
170 |
0.0079 |
- |
- |
- |
- |
| 3.5079 |
180 |
0.0156 |
- |
- |
- |
- |
| 3.7028 |
190 |
0.0075 |
- |
- |
- |
- |
| 3.8977 |
200 |
0.0254 |
- |
- |
- |
- |
| 3.9756 |
204 |
- |
0.3947 |
0.3999 |
0.3882 |
0.3705 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.11.10
- Sentence Transformers: 3.2.1
- Transformers: 4.43.1
- PyTorch: 2.5.1+cu124
- Accelerate: 1.0.1
- Datasets: 3.0.2
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
