metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:36
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-m-v1.5
widget:
- source_sentence: >-
What are the abstract methods provided for managing model servers in
ZenML's BaseModelDeployerFlavor class?
sentences:
- >-
quired for your GCP Image Builder by running e.g.:zenml
service-connector list-resources --resource-type gcp-generic
Example Command Output
The following 'gcp-generic' resources can be accessed by service
connectors that you have configured:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE
│ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼────────────────┼────────────────┼────────────────┼────────────────┨
┃ bfdb657d-d808-47e7-9974-9ba6e4919d83 │ gcp-generic │ 🔵 gcp
│ 🔵 gcp-generic │ zenml-core ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛
After having set up or decided on a GCP Service Connector to use to
authenticate to GCP, you can register the GCP Image Builder as follows:
zenml image-builder register <IMAGE_BUILDER_NAME> \
--flavor=gcp \
--cloud_builder_image=<BUILDER_IMAGE_NAME> \
--network=<DOCKER_NETWORK> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS>
zenml image-builder connect <IMAGE_BUILDER_NAME> -i
A non-interactive version that connects the GCP Image Builder to a
target GCP Service Connector:
zenml image-builder connect <IMAGE_BUILDER_NAME> --connector
<CONNECTOR_ID>
Example Command Output
- >-
🧙Installation
Installing ZenML and getting started.
ZenML is a Python package that can be installed directly via pip:
pip install zenml
Note that ZenML currently supports Python 3.8, 3.9, 3.10, and 3.11.
Please make sure that you are using a supported Python version.
Install with the dashboard
ZenML comes bundled with a web dashboard that lives inside a sister
repository. In order to get access to the dashboard locally, you need to
launch the ZenML Server and Dashboard locally. For this, you need to
install the optional dependencies for the ZenML Server:
pip install "zenml[server]"
We highly encourage you to install ZenML in a virtual environment. At
ZenML, We like to use virtualenvwrapper or pyenv-virtualenv to manage
our Python virtual environments.
Installing onto MacOS with Apple Silicon (M1, M2)
A change in how forking works on Macs running on Apple Silicon means
that you should set the following environment variable which will ensure
that your connections to the server remain unbroken:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
You can read more about this here. This environment variable is needed
if you are working with a local server on your Mac, but if you're just
using ZenML as a client / CLI and connecting to a deployed server then
you don't need to set it.
Nightly builds
ZenML also publishes nightly builds under the zenml-nightly package
name. These are built from the latest develop branch (to which work
ready for release is published) and are not guaranteed to be stable. To
install the nightly build, run:
pip install zenml-nightly
Verifying installations
Once the installation is completed, you can check whether the
installation was successful either through Bash:
zenml version
or through Python:
import zenml
print(zenml.__version__)
If you would like to learn more about the current release, please visit
our PyPi package page.
Running with Docker
- >-
:
"""Abstract method to deploy a model."""@staticmethod
@abstractmethod
def get_model_server_info(
service: BaseService,
) -> Dict[str, Optional[str]]:
"""Give implementation-specific way to extract relevant model server
properties for the user."""
@abstractmethod
def perform_stop_model(
self,
service: BaseService,
timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,
force: bool = False,
) -> BaseService:
"""Abstract method to stop a model server."""
@abstractmethod
def perform_start_model(
self,
service: BaseService,
timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,
) -> BaseService:
"""Abstract method to start a model server."""
@abstractmethod
def perform_delete_model(
self,
service: BaseService,
timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,
force: bool = False,
) -> None:
"""Abstract method to delete a model server."""
class BaseModelDeployerFlavor(Flavor):
"""Base class for model deployer flavors."""
@property
@abstractmethod
def name(self):
"""Returns the name of the flavor."""
@property
def type(self) -> StackComponentType:
"""Returns the flavor type.
Returns:
The flavor type.
"""
return StackComponentType.MODEL_DEPLOYER
@property
def config_class(self) -> Type[BaseModelDeployerConfig]:
"""Returns `BaseModelDeployerConfig` config class.
Returns:
The config class.
"""
return BaseModelDeployerConfig
@property
@abstractmethod
def implementation_class(self) -> Type[BaseModelDeployer]:
"""The class that implements the model deployer."""
This is a slimmed-down version of the base implementation which aims to
highlight the abstraction layer. In order to see the full implementation
and get the complete docstrings, please check the SDK docs .
Building your own model deployers
- source_sentence: >-
How can you successfully connect the image builder `gcp-image-builder` to
the resources using a connector ID?
sentences:
- |-
ZenML - Bridging the gap between ML & Ops
Legacy Docs
Bleeding EdgeLegacy Docs0.67.0
🧙♂️Find older version our docs
Powered by GitBook
- >-
Google Cloud Image Builder
Building container images with Google Cloud Build
The Google Cloud image builder is an image builder flavor provided by
the ZenML gcp integration that uses Google Cloud Build to build
container images.
When to use it
You should use the Google Cloud image builder if:
you're unable to install or use Docker on your client machine.
you're already using GCP.
your stack is mainly composed of other Google Cloud components such as
the GCS Artifact Store or the Vertex Orchestrator.
How to deploy it
Would you like to skip ahead and deploy a full ZenML cloud stack
already, including the Google Cloud image builder? Check out the
in-browser stack deployment wizard, the stack registration wizard, or
the ZenML GCP Terraform module for a shortcut on how to deploy &
register this stack component.
In order to use the ZenML Google Cloud image builder you need to enable
Google Cloud Build relevant APIs on the Google Cloud project.
How to use it
To use the Google Cloud image builder, we need:
The ZenML gcp integration installed. If you haven't done so, run:
zenml integration install gcp
A GCP Artifact Store where the build context will be uploaded, so Google
Cloud Build can access it.
A GCP container registry where the built image will be pushed.
Optionally, the GCP project ID in which you want to run the build and a
service account with the needed permissions to run the build. If not
provided, then the project ID and credentials will be inferred from the
environment.
Optionally, you can change:
the Docker image used by Google Cloud Build to execute the steps to
build and push the Docker image. By default, the builder image will be
'gcr.io/cloud-builders/docker'.
The network to which the container used to build the ZenML pipeline
Docker image will be attached. More information: Cloud build network.
The build timeout for the build, and for the blocking operation waiting
for the build to finish. More information: Build Timeout.
- >-
--connector <CONNECTOR_ID>
Example Command Output$ zenml image-builder connect gcp-image-builder
--connector gcp-generic
Successfully connected image builder `gcp-image-builder` to the
following resources:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE
│ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼────────────────┼────────────────┼────────────────┼────────────────┨
┃ bfdb657d-d808-47e7-9974-9ba6e4919d83 │ gcp-generic │ 🔵 gcp
│ 🔵 gcp-generic │ zenml-core ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛
As a final step, you can use the GCP Image Builder in a ZenML Stack:
zenml stack register <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set
When you register the GCP Image Builder, you can generate a GCP Service
Account Key, save it to a local file and then reference it in the Image
Builder configuration.
This method has the advantage that you don't need to install and
configure the GCP CLI on your host, but it's still not as secure as
using a GCP Service Connector and the stack component configuration is
not portable to other hosts.
For this method, you need to create a user-managed GCP service account,
and grant it privileges to access the Cloud Build API and to run Cloud
Builder jobs (e.g. the Cloud Build Editor IAM role.
With the service account key downloaded to a local file, you can
register the GCP Image Builder as follows:
zenml image-builder register <IMAGE_BUILDER_NAME> \
--flavor=gcp \
--project=<GCP_PROJECT_ID> \
--service_account_path=<PATH_TO_SERVICE_ACCOUNT_KEY> \
--cloud_builder_image=<BUILDER_IMAGE_NAME> \
--network=<DOCKER_NETWORK> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS>
- source_sentence: How do I finetune embeddings using Sentence Transformers in ZenML?
sentences:
- >-
nsible for cluster-manager-specific configuration._io_configuration is a
critical method. Even though we have materializers, Spark might require
additional packages and configuration to work with a specific
filesystem. This method is used as an interface to provide this
configuration.
_additional_configuration takes the submit_args, converts, and appends
them to the overall configuration.
Once the configuration is completed, _launch_spark_job comes into play.
This takes the completed configuration and runs a Spark job on the given
master URL with the specified deploy_mode. By default, this is achieved
by creating and executing a spark-submit command.
Warning
In its first iteration, the pre-configuration with _io_configuration
method is only effective when it is paired with an S3ArtifactStore
(which has an authentication secret). When used with other artifact
store flavors, you might be required to provide additional configuration
through the submit_args.
Stack Component: KubernetesSparkStepOperator
The KubernetesSparkStepOperator is implemented by subclassing the base
SparkStepOperator and uses the PipelineDockerImageBuilder class to build
and push the required Docker images.
from typing import Optional
from zenml.integrations.spark.step_operators.spark_step_operator import
(
SparkStepOperatorConfig
)
class KubernetesSparkStepOperatorConfig(SparkStepOperatorConfig):
"""Config for the Kubernetes Spark step operator."""
namespace: Optional[str] = None
service_account: Optional[str] = None
from pyspark.conf import SparkConf
from zenml.utils.pipeline_docker_image_builder import
PipelineDockerImageBuilder
from zenml.integrations.spark.step_operators.spark_step_operator import
(
SparkStepOperator
)
class KubernetesSparkStepOperator(SparkStepOperator):
"""Step operator which runs Steps with Spark on Kubernetes."""
- |-
Finetuning embeddings with Sentence Transformers
Finetune embeddings with Sentence Transformers.
PreviousSynthetic data generationNextEvaluating finetuned embeddings
Last updated 1 month ago
- >-
Whylogs
How to collect and visualize statistics to track changes in your
pipelines' data with whylogs/WhyLabs profiling.
The whylogs/WhyLabs Data Validator flavor provided with the ZenML
integration uses whylogs and WhyLabs to generate and track data
profiles, highly accurate descriptive representations of your data. The
profiles can be used to implement automated corrective actions in your
pipelines, or to render interactive representations for further visual
interpretation, evaluation and documentation.
When would you want to use it?
Whylogs is an open-source library that analyzes your data and creates
statistical summaries called whylogs profiles. Whylogs profiles can be
processed in your pipelines and visualized locally or uploaded to the
WhyLabs platform, where more in depth analysis can be carried out. Even
though whylogs also supports other data types, the ZenML whylogs
integration currently only works with tabular data in pandas.DataFrame
format.
You should use the whylogs/WhyLabs Data Validator when you need the
following data validation features that are possible with whylogs and
WhyLabs:
Data Quality: validate data quality in model inputs or in a data
pipeline
Data Drift: detect data drift in model input features
Model Drift: Detect training-serving skew, concept drift, and model
performance degradation
You should consider one of the other Data Validator flavors if you need
a different set of data validation features.
How do you deploy it?
The whylogs Data Validator flavor is included in the whylogs ZenML
integration, you need to install it on your local machine to be able to
register a whylogs Data Validator and add it to your stack:
zenml integration install whylogs -y
If you don't need to connect to the WhyLabs platform to upload and store
the generated whylogs data profiles, the Data Validator stack component
does not require any configuration parameters. Adding it to a stack is
as simple as running e.g.:
- source_sentence: What is the purpose of ZenML in the context of ML and Ops?
sentences:
- >-
> \
--build_timeout=<BUILD_TIMEOUT_IN_SECONDS># Register and set a stack with the new image builder
zenml stack register <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set
Caveats
As described in this Google Cloud Build documentation page, Google Cloud
Build uses containers to execute the build steps which are automatically
attached to a network called cloudbuild that provides some Application
Default Credentials (ADC), that allow the container to be authenticated
and therefore use other GCP services.
By default, the GCP Image Builder is executing the build command of the
ZenML Pipeline Docker image with the option --network=cloudbuild, so the
ADC provided by the cloudbuild network can also be used in the build.
This is useful if you want to install a private dependency from a GCP
Artifact Registry, but you will also need to use a custom base parent
image with the keyrings.google-artifactregistry-auth installed, so pip
can connect and authenticate in the private artifact registry to
download the dependency.
FROM zenmldocker/zenml:latest
RUN pip install keyrings.google-artifactregistry-auth
The above Dockerfile uses zenmldocker/zenml:latest as a base image, but
is recommended to change the tag to specify the ZenML version and Python
version like 0.33.0-py3.10.
PreviousKaniko Image BuilderNextDevelop a Custom Image Builder
Last updated 21 days ago
- >-
Control caching behavior
By default steps in ZenML pipelines are cached whenever code and
parameters stay unchanged.
@step(enable_cache=True)
def load_data(parameter: int) -> dict:
...
@step(enable_cache=False)
level
def train_model(data: dict) -> None:
...
@pipeline(enable_cache=True)
def simple_ml_pipeline(parameter: int):
...
Caching only happens when code and parameters stay the same.
Like many other step and pipeline settings, you can also change this
afterward:
my_step.configure(enable_cache=...)
my_pipeline.configure(enable_cache=...)
Find out here how to configure this in a YAML file
PreviousStep output typing and annotationNextSchedule a pipeline
Last updated 4 months ago
- |-
ZenML - Bridging the gap between ML & Ops
Legacy Docs
Bleeding EdgeLegacy Docs0.67.0
🧙♂️Find older version our docs
Powered by GitBook
- source_sentence: How does ZenML facilitate the flow of data between steps in a pipeline?
sentences:
- >-
tainer_registry \
-i local_builder \
--setOnce you added the step operator to your active stack, you can use it to execute individual steps of your pipeline by specifying it in the @step decorator as follows:
from zenml import step
@step(step_operator=<STEP_OPERATOR_NAME>)
def step_on_spark(...) -> ...:
"""Some step that should run with Spark on Kubernetes."""
...
After successfully running any step with a KubernetesSparkStepOperator,
you should be able to see that a Spark driver pod was created in your
cluster for each pipeline step when running kubectl get pods -n
$KUBERNETES_NAMESPACE.
Instead of hardcoding a step operator name, you can also use the Client
to dynamically use the step operator of your active stack:
from zenml.client import Client
step_operator = Client().active_stack.step_operator
@step(step_operator=step_operator.name)
def step_on_spark(...) -> ...:
...
Additional configuration
For additional configuration of the Spark step operator, you can pass
SparkStepOperatorSettings when defining or running your pipeline. Check
out the SDK docs for a full list of available attributes and this docs
page for more information on how to specify settings.
PreviousKubernetesNextDevelop a Custom Step Operator
Last updated 4 months ago
- >-
🗄️Handle Data/Artifacts
Step outputs in ZenML are stored in the artifact store. This enables
caching, lineage and auditability. Using type annotations helps with
transparency, passing data between steps, and serializing/des
For best results, use type annotations for your outputs. This is good
coding practice for transparency, helps ZenML handle passing data
between steps, and also enables ZenML to serialize and deserialize
(referred to as 'materialize' in ZenML) the data.
@step
def load_data(parameter: int) -> Dict[str, Any]:
training_data = [[1, 2], [3, 4], [5, 6]]
labels = [0, 1, 0]
return {'features': training_data, 'labels': labels}
@step
def train_model(data: Dict[str, Any]) -> None:
total_features = sum(map(sum, data['features']))
total_labels = sum(data['labels'])
print(f"Trained model using {len(data['features'])} data points. "
f"Feature sum is {total_features}, label sum is {total_labels}")
@pipeline
def simple_ml_pipeline(parameter: int):
dataset = load_data(parameter=parameter) # Get the output
train_model(dataset) # Pipe the previous step output into the downstream step
In this code, we define two steps: load_data and train_model. The
load_data step takes an integer parameter and returns a dictionary
containing training data and labels. The train_model step receives the
dictionary from load_data, extracts the features and labels, and trains
a model (not shown here).
Finally, we define a pipeline simple_ml_pipeline that chains the
load_data and train_model steps together. The output from load_data is
passed as input to train_model, demonstrating how data flows between
steps in a ZenML pipeline.
PreviousDisable colorful loggingNextHow ZenML stores data
Last updated 4 months ago
- >-
in the ZenML dashboard.
The whylogs standard stepZenML wraps the whylogs/WhyLabs functionality
in the form of a standard WhylogsProfilerStep step. The only field in
the step config is a dataset_timestamp attribute which is only relevant
when you upload the profiles to WhyLabs that uses this field to group
and merge together profiles belonging to the same dataset. The helper
function get_whylogs_profiler_step used to create an instance of this
standard step takes in an optional dataset_id parameter that is also
used only in the context of WhyLabs upload to identify the model in the
context of which the profile is uploaded, e.g.:
from zenml.integrations.whylogs.steps import get_whylogs_profiler_step
train_data_profiler = get_whylogs_profiler_step(dataset_id="model-2")
test_data_profiler = get_whylogs_profiler_step(dataset_id="model-3")
The step can then be inserted into your pipeline where it can take in a
pandas.DataFrame dataset, e.g.:
from zenml import pipeline
@pipeline
def data_profiling_pipeline():
data, _ = data_loader()
train, test = data_splitter(data)
train_data_profiler(train)
test_data_profiler(test)
data_profiling_pipeline()
As can be seen from the step definition , the step takes in a dataset
and returns a whylogs DatasetProfileView object:
@step
def whylogs_profiler_step(
dataset: pd.DataFrame,
dataset_timestamp: Optional[datetime.datetime] = None,
) -> DatasetProfileView:
...
You should consult the official whylogs documentation for more
information on what you can do with the collected profiles.
You can view the complete list of configuration parameters in the SDK
docs.
The whylogs Data Validator
The whylogs Data Validator implements the same interface as do all Data
Validators, so this method forces you to maintain some level of
compatibility with the overall Data Validator abstraction, which
guarantees an easier migration in case you decide to switch to another
Data Validator.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: zenml/finetuned-snowflake-arctic-embed-m-v1.5
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 1
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 1
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 1
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 1
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 1
name: Cosine Mrr@10
- type: cosine_map@100
value: 1
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 1
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 1
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 1
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 1
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 1
name: Cosine Mrr@10
- type: cosine_map@100
value: 1
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 1
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 1
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 1
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 1
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 1
name: Cosine Mrr@10
- type: cosine_map@100
value: 1
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.75
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.75
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.2
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.1
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.75
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9077324383928644
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.875
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.875
name: Cosine Map@100
zenml/finetuned-snowflake-arctic-embed-m-v1.5
This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: Snowflake/snowflake-arctic-embed-m-v1.5
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5")
sentences = [
'How does ZenML facilitate the flow of data between steps in a pipeline?',
'🗄️Handle Data/Artifacts\n\nStep outputs in ZenML are stored in the artifact store. This enables caching, lineage and auditability. Using type annotations helps with transparency, passing data between steps, and serializing/des\n\nFor best results, use type annotations for your outputs. This is good coding practice for transparency, helps ZenML handle passing data between steps, and also enables ZenML to serialize and deserialize (referred to as \'materialize\' in ZenML) the data.\n\n@step\ndef load_data(parameter: int) -> Dict[str, Any]:\n\n# do something with the parameter here\n\ntraining_data = [[1, 2], [3, 4], [5, 6]]\n labels = [0, 1, 0]\n return {\'features\': training_data, \'labels\': labels}\n\n@step\ndef train_model(data: Dict[str, Any]) -> None:\n total_features = sum(map(sum, data[\'features\']))\n total_labels = sum(data[\'labels\'])\n \n # Train some model here\n \n print(f"Trained model using {len(data[\'features\'])} data points. "\n f"Feature sum is {total_features}, label sum is {total_labels}")\n\n@pipeline \ndef simple_ml_pipeline(parameter: int):\n dataset = load_data(parameter=parameter) # Get the output \n train_model(dataset) # Pipe the previous step output into the downstream step\n\nIn this code, we define two steps: load_data and train_model. The load_data step takes an integer parameter and returns a dictionary containing training data and labels. The train_model step receives the dictionary from load_data, extracts the features and labels, and trains a model (not shown here).\n\nFinally, we define a pipeline simple_ml_pipeline that chains the load_data and train_model steps together. The output from load_data is passed as input to train_model, demonstrating how data flows between steps in a ZenML pipeline.\n\nPreviousDisable colorful loggingNextHow ZenML stores data\n\nLast updated 4 months ago',
'in the ZenML dashboard.\n\nThe whylogs standard stepZenML wraps the whylogs/WhyLabs functionality in the form of a standard WhylogsProfilerStep step. The only field in the step config is a dataset_timestamp attribute which is only relevant when you upload the profiles to WhyLabs that uses this field to group and merge together profiles belonging to the same dataset. The helper function get_whylogs_profiler_step used to create an instance of this standard step takes in an optional dataset_id parameter that is also used only in the context of WhyLabs upload to identify the model in the context of which the profile is uploaded, e.g.:\n\nfrom zenml.integrations.whylogs.steps import get_whylogs_profiler_step\n\ntrain_data_profiler = get_whylogs_profiler_step(dataset_id="model-2")\ntest_data_profiler = get_whylogs_profiler_step(dataset_id="model-3")\n\nThe step can then be inserted into your pipeline where it can take in a pandas.DataFrame dataset, e.g.:\n\nfrom zenml import pipeline\n\n@pipeline\ndef data_profiling_pipeline():\n data, _ = data_loader()\n train, test = data_splitter(data)\n train_data_profiler(train)\n test_data_profiler(test)\n\ndata_profiling_pipeline()\n\nAs can be seen from the step definition , the step takes in a dataset and returns a whylogs DatasetProfileView object:\n\n@step\ndef whylogs_profiler_step(\n dataset: pd.DataFrame,\n dataset_timestamp: Optional[datetime.datetime] = None,\n) -> DatasetProfileView:\n ...\n\nYou should consult the official whylogs documentation for more information on what you can do with the collected profiles.\n\nYou can view the complete list of configuration parameters in the SDK docs.\n\nThe whylogs Data Validator\n\nThe whylogs Data Validator implements the same interface as do all Data Validators, so this method forces you to maintain some level of compatibility with the overall Data Validator abstraction, which guarantees an easier migration in case you decide to switch to another Data Validator.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
1.0 |
| cosine_accuracy@3 |
1.0 |
| cosine_accuracy@5 |
1.0 |
| cosine_accuracy@10 |
1.0 |
| cosine_precision@1 |
1.0 |
| cosine_precision@3 |
0.3333 |
| cosine_precision@5 |
0.2 |
| cosine_precision@10 |
0.1 |
| cosine_recall@1 |
1.0 |
| cosine_recall@3 |
1.0 |
| cosine_recall@5 |
1.0 |
| cosine_recall@10 |
1.0 |
| cosine_ndcg@10 |
1.0 |
| cosine_mrr@10 |
1.0 |
| cosine_map@100 |
1.0 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
1.0 |
| cosine_accuracy@3 |
1.0 |
| cosine_accuracy@5 |
1.0 |
| cosine_accuracy@10 |
1.0 |
| cosine_precision@1 |
1.0 |
| cosine_precision@3 |
0.3333 |
| cosine_precision@5 |
0.2 |
| cosine_precision@10 |
0.1 |
| cosine_recall@1 |
1.0 |
| cosine_recall@3 |
1.0 |
| cosine_recall@5 |
1.0 |
| cosine_recall@10 |
1.0 |
| cosine_ndcg@10 |
1.0 |
| cosine_mrr@10 |
1.0 |
| cosine_map@100 |
1.0 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
1.0 |
| cosine_accuracy@3 |
1.0 |
| cosine_accuracy@5 |
1.0 |
| cosine_accuracy@10 |
1.0 |
| cosine_precision@1 |
1.0 |
| cosine_precision@3 |
0.3333 |
| cosine_precision@5 |
0.2 |
| cosine_precision@10 |
0.1 |
| cosine_recall@1 |
1.0 |
| cosine_recall@3 |
1.0 |
| cosine_recall@5 |
1.0 |
| cosine_recall@10 |
1.0 |
| cosine_ndcg@10 |
1.0 |
| cosine_mrr@10 |
1.0 |
| cosine_map@100 |
1.0 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.75 |
| cosine_accuracy@3 |
1.0 |
| cosine_accuracy@5 |
1.0 |
| cosine_accuracy@10 |
1.0 |
| cosine_precision@1 |
0.75 |
| cosine_precision@3 |
0.3333 |
| cosine_precision@5 |
0.2 |
| cosine_precision@10 |
0.1 |
| cosine_recall@1 |
0.75 |
| cosine_recall@3 |
1.0 |
| cosine_recall@5 |
1.0 |
| cosine_recall@10 |
1.0 |
| cosine_ndcg@10 |
0.9077 |
| cosine_mrr@10 |
0.875 |
| cosine_map@100 |
0.875 |
Training Details
Training Dataset
json
- Dataset: json
- Size: 36 training samples
- Columns:
positive and anchor
- Approximate statistics based on the first 36 samples:
|
positive |
anchor |
| type |
string |
string |
| details |
- min: 13 tokens
- mean: 23.92 tokens
- max: 41 tokens
|
- min: 31 tokens
- mean: 321.11 tokens
- max: 512 tokens
|
- Samples:
| positive |
anchor |
How does ZenML integrate Spark step operators for executing individual steps? |
Spark
Executing individual steps on Spark
The spark integration brings two different step operators:
Step Operator: The SparkStepOperator serves as the base class for all the Spark-related step operators.
Step Operator: The KubernetesSparkStepOperator is responsible for launching ZenML steps as Spark applications with Kubernetes as a cluster manager.
Step Operators: SparkStepOperator
A summarized version of the implementation can be summarized in two parts. First, the configuration:
from typing import Optional, Dict, Any
from zenml.step_operators import BaseStepOperatorConfig
class SparkStepOperatorConfig(BaseStepOperatorConfig):
"""Spark step operator config.
Attributes:
master: is the master URL for the cluster. You might see different
schemes for different cluster managers which are supported by Spark
like Mesos, YARN, or Kubernetes. Within the context of this PR,
the implementation supports Kubernetes as a cluster manager.
deploy_mode: can either be 'cluster' (default) or 'client' and it
decides where the driver node of the application will run.
submit_kwargs: is the JSON string of a dict, which will be used
to define additional params if required (Spark has quite a
lot of different parameters, so including them, all in the step
operator was not implemented).
"""
master: str
deploy_mode: str = "cluster"
submit_kwargs: Optional[Dict[str, Any]] = None
and then the implementation:
from typing import List
from pyspark.conf import SparkConf
from zenml.step_operators import BaseStepOperator
class SparkStepOperator(BaseStepOperator):
"""Base class for all Spark-related step operators."""
def _resource_configuration(
self,
spark_config: SparkConf,
resource_configuration: "ResourceSettings",
) -> None:
"""Configures Spark to handle the resource configuration.""" |
How can ZenML be used to finetune LLMs for specific tasks or to improve performance and cost? |
Finetuning LLMs with ZenML
Finetune LLMs for specific tasks or to improve performance and cost.
PreviousEvaluating finetuned embeddingsNextSet up a project repository
Last updated 6 months ago |
How can I develop a custom model deployer in ZenML for efficient deployment and management of machine-learning models? |
Develop a Custom Model Deployer
Learning how to develop a custom model deployer.
Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML's component flavor concepts.
To deploy and manage your trained machine-learning models, ZenML provides a stack component called Model Deployer. This component is responsible for interacting with the deployment tool, framework, or platform.
When present in a stack, the model deployer can also act as a registry for models that are served with ZenML. You can use the model deployer to list all models that are currently deployed for online inference or filtered according to a particular pipeline run or step, or to suspend, resume or delete an external model server managed through ZenML.
Base Abstraction
In ZenML, the base abstraction of the model deployer is built on top of three major criteria:
It needs to ensure efficient deployment and management of models in accordance with the specific requirements of the serving infrastructure, by holding all the stack-related configuration attributes required to interact with the remote model serving tool, service, or platform.
It needs to implement the continuous deployment logic necessary to deploy models in a way that updates an existing model server that is already serving a previous version of the same model instead of creating a new model server for every new model version (see the deploy_model abstract method). This functionality can be consumed directly from ZenML pipeline steps, but it can also be used outside the pipeline to deploy ad-hoc models. It is also usually coupled with a standard model deployer step, implemented by each integration, that hides the details of the deployment process from the user. |
- Loss:
MatryoshkaLoss with these parameters:{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
384,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1
],
"n_dims_per_step": -1
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: epochper_device_train_batch_size: 4per_device_eval_batch_size: 16gradient_accumulation_steps: 16learning_rate: 2e-05num_train_epochs: 4lr_scheduler_type: cosinewarmup_ratio: 0.1tf32: Falseload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 4per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 16eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 4max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Falsefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Falselocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseeval_use_gather_object: Falsebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch |
Step |
dim_384_cosine_map@100 |
dim_256_cosine_map@100 |
dim_128_cosine_map@100 |
dim_64_cosine_map@100 |
| 1.0 |
1 |
1.0 |
1.0 |
0.875 |
0.875 |
| 2.0 |
3 |
1.0 |
1.0 |
1.0 |
0.875 |
| 3.0 |
4 |
1.0 |
1.0 |
1.0 |
0.875 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.11.10
- Sentence Transformers: 3.2.1
- Transformers: 4.43.1
- PyTorch: 2.5.1+cu124
- Accelerate: 1.0.1
- Datasets: 3.0.2
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
