--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:36 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: Snowflake/snowflake-arctic-embed-m-v1.5 widget: - source_sentence: Where can I find older versions of ZenML documentation? sentences: - 'πŸ§™Installation Installing ZenML and getting started. ZenML is a Python package that can be installed directly via pip: pip install zenml Note that ZenML currently supports Python 3.8, 3.9, 3.10, and 3.11. Please make sure that you are using a supported Python version. Install with the dashboard ZenML comes bundled with a web dashboard that lives inside a sister repository. In order to get access to the dashboard locally, you need to launch the ZenML Server and Dashboard locally. For this, you need to install the optional dependencies for the ZenML Server: pip install "zenml[server]" We highly encourage you to install ZenML in a virtual environment. At ZenML, We like to use virtualenvwrapper or pyenv-virtualenv to manage our Python virtual environments. Installing onto MacOS with Apple Silicon (M1, M2) A change in how forking works on Macs running on Apple Silicon means that you should set the following environment variable which will ensure that your connections to the server remain unbroken: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES You can read more about this here. This environment variable is needed if you are working with a local server on your Mac, but if you''re just using ZenML as a client / CLI and connecting to a deployed server then you don''t need to set it. Nightly builds ZenML also publishes nightly builds under the zenml-nightly package name. These are built from the latest develop branch (to which work ready for release is published) and are not guaranteed to be stable. To install the nightly build, run: pip install zenml-nightly Verifying installations Once the installation is completed, you can check whether the installation was successful either through Bash: zenml version or through Python: import zenml print(zenml.__version__) If you would like to learn more about the current release, please visit our PyPi package page. Running with Docker' - 'Finetuning LLMs with ZenML Finetune LLMs for specific tasks or to improve performance and cost. PreviousEvaluating finetuned embeddingsNextSet up a project repository Last updated 6 months ago' - 'ZenML - Bridging the gap between ML & Ops Legacy Docs Bleeding EdgeLegacy Docs0.67.0 πŸ§™β€β™‚οΈFind older version our docs Powered by GitBook' - source_sentence: How can you visualize the profiles generated by whylogs in ZenML? sentences: - 'Whylogs How to collect and visualize statistics to track changes in your pipelines'' data with whylogs/WhyLabs profiling. The whylogs/WhyLabs Data Validator flavor provided with the ZenML integration uses whylogs and WhyLabs to generate and track data profiles, highly accurate descriptive representations of your data. The profiles can be used to implement automated corrective actions in your pipelines, or to render interactive representations for further visual interpretation, evaluation and documentation. When would you want to use it? Whylogs is an open-source library that analyzes your data and creates statistical summaries called whylogs profiles. Whylogs profiles can be processed in your pipelines and visualized locally or uploaded to the WhyLabs platform, where more in depth analysis can be carried out. Even though whylogs also supports other data types, the ZenML whylogs integration currently only works with tabular data in pandas.DataFrame format. You should use the whylogs/WhyLabs Data Validator when you need the following data validation features that are possible with whylogs and WhyLabs: Data Quality: validate data quality in model inputs or in a data pipeline Data Drift: detect data drift in model input features Model Drift: Detect training-serving skew, concept drift, and model performance degradation You should consider one of the other Data Validator flavors if you need a different set of data validation features. How do you deploy it? The whylogs Data Validator flavor is included in the whylogs ZenML integration, you need to install it on your local machine to be able to register a whylogs Data Validator and add it to your stack: zenml integration install whylogs -y If you don''t need to connect to the WhyLabs platform to upload and store the generated whylogs data profiles, the Data Validator stack component does not require any configuration parameters. Adding it to a stack is as simple as running e.g.:' - "Control caching behavior\n\nBy default steps in ZenML pipelines are cached whenever\ \ code and parameters stay unchanged.\n\n@step(enable_cache=True) # set cache\ \ behavior at step level\ndef load_data(parameter: int) -> dict:\n ...\n\n\ @step(enable_cache=False) # settings at step level override pipeline level\ndef\ \ train_model(data: dict) -> None:\n ...\n\n@pipeline(enable_cache=True) #\ \ set cache behavior at step level\ndef simple_ml_pipeline(parameter: int):\n\ \ ...\n\nCaching only happens when code and parameters stay the same.\n\nLike\ \ many other step and pipeline settings, you can also change this afterward:\n\ \n# Same as passing it in the step decorator\nmy_step.configure(enable_cache=...)\n\ \n# Same as passing it in the pipeline decorator\nmy_pipeline.configure(enable_cache=...)\n\ \nFind out here how to configure this in a YAML file\n\nPreviousStep output typing\ \ and annotationNextSchedule a pipeline\n\nLast updated 4 months ago" - "ogsDataValidatorSettings,\n)\nfrom zenml import step@step(\n settings={\n\ \ \"data_validator\": WhylogsDataValidatorSettings(\n enable_whylabs=True,\ \ dataset_id=\"model-1\"\n )\n }\n)\ndef data_loader() -> Tuple[\n \ \ Annotated[pd.DataFrame, \"data\"],\n Annotated[DatasetProfileView, \"profile\"\ ]\n]:\n \"\"\"Load the diabetes dataset.\"\"\"\n X, y = datasets.load_diabetes(return_X_y=True,\ \ as_frame=True)\n\n# merge X and y together\n df = pd.merge(X, y, left_index=True,\ \ right_index=True)\n\nprofile = why.log(pandas=df).profile().view()\n return\ \ df, profile\n\nHow do you use it?\n\nWhylogs's profiling functions take in a\ \ pandas.DataFrame dataset generate a DatasetProfileView object containing all\ \ the relevant information extracted from the dataset.\n\nThere are three ways\ \ you can use whylogs in your ZenML pipelines that allow different levels of flexibility:\n\ \ninstantiate, configure and insert the standard WhylogsProfilerStep shipped with\ \ ZenML into your pipelines. This is the easiest way and the recommended approach,\ \ but can only be customized through the supported step configuration parameters.\n\ \ncall the data validation methods provided by the whylogs Data Validator in your\ \ custom step implementation. This method allows for more flexibility concerning\ \ what can happen in the pipeline step, but you are still limited to the functionality\ \ implemented in the Data Validator.\n\nuse the whylogs library directly in your\ \ custom step implementation. This gives you complete freedom in how you are using\ \ whylogs's features.\n\nYou can visualize whylogs profiles in Jupyter notebooks\ \ or view them directly in the ZenML dashboard.\n\nThe whylogs standard step" - source_sentence: How can I build my own custom experiment tracker flavor in ZenML? sentences: - "e details of the deployment process from the user.It needs to act as a ZenML\ \ BaseService registry, where every BaseService instance is used as an internal\ \ representation of a remote model server (see the find_model_server abstract\ \ method). To achieve this, it must be able to re-create the configuration of\ \ a BaseService from information that is persisted externally, alongside, or even\ \ as part of the remote model server configuration itself. For example, for model\ \ servers that are implemented as Kubernetes resources, the BaseService instances\ \ can be serialized and saved as Kubernetes resource annotations. This allows\ \ the model deployer to keep track of all externally running model servers and\ \ to re-create their corresponding BaseService instance representations at any\ \ given time. The model deployer also defines methods that implement basic life-cycle\ \ management on remote model servers outside the coverage of a pipeline (see stop_model_server\ \ , start_model_server and delete_model_server).\n\nPutting all these considerations\ \ together, we end up with the following interface:\n\nfrom abc import ABC, abstractmethod\n\ from typing import Dict, List, Optional, Type\nfrom uuid import UUID\n\nfrom zenml.enums\ \ import StackComponentType\nfrom zenml.services import BaseService, ServiceConfig\n\ from zenml.stack import StackComponent, StackComponentConfig, Flavor\n\nDEFAULT_DEPLOYMENT_START_STOP_TIMEOUT\ \ = 300\n\nclass BaseModelDeployerConfig(StackComponentConfig):\n \"\"\"Base\ \ class for all ZenML model deployer configurations.\"\"\"\n\nclass BaseModelDeployer(StackComponent,\ \ ABC):\n \"\"\"Base class for all ZenML model deployers.\"\"\"\n\n@abstractmethod\n\ \ def perform_deploy_model(\n self,\n id: UUID,\n config:\ \ ServiceConfig,\n timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n\ \ ) -> BaseService:\n \"\"\"Abstract method to deploy a model.\"\"\"" - 'Develop a custom experiment tracker Learning how to develop a custom experiment tracker. Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML''s component flavor concepts. Base abstraction in progress! We are actively working on the base abstraction for the Experiment Tracker, which will be available soon. As a result, their extension is not recommended at the moment. When you are selecting an Experiment Tracker for your stack, you can use one of the existing flavors. If you need to implement your own Experiment Tracker flavor, you can still do so, but keep in mind that you may have to refactor it when the base abstraction is released. Build your own custom experiment tracker If you want to create your own custom flavor for an experiment tracker, you can follow the following steps: Create a class that inherits from the BaseExperimentTracker class and implements the abstract methods. If you need any configuration, create a class that inherits from the BaseExperimentTrackerConfig class and add your configuration parameters. Bring both the implementation and the configuration together by inheriting from the BaseExperimentTrackerFlavor class. Once you are done with the implementation, you can register it through the CLI. Please ensure you point to the flavor class via dot notation: zenml experiment-tracker flavor register For example, if your flavor class MyExperimentTrackerFlavor is defined in flavors/my_flavor.py, you''d register it by doing: zenml experiment-tracker flavor register flavors.my_flavor.MyExperimentTrackerFlavor' - "res Spark to handle the resource configuration.\"\"\"def _backend_configuration(\n\ \ self,\n spark_config: SparkConf,\n step_config:\ \ \"StepConfiguration\",\n ) -> None:\n \"\"\"Configures Spark to handle\ \ backends like YARN, Mesos or Kubernetes.\"\"\"\n\ndef _io_configuration(\n \ \ self,\n spark_config: SparkConf\n ) -> None:\n \ \ \"\"\"Configures Spark to handle different input/output sources.\"\"\"\n\n\ def _additional_configuration(\n self,\n spark_config: SparkConf\n\ \ ) -> None:\n \"\"\"Appends the user-defined configuration parameters.\"\ \"\"\n\ndef _launch_spark_job(\n self,\n spark_config: SparkConf,\n\ \ entrypoint_command: List[str]\n ) -> None:\n \"\"\"Generates\ \ and executes a spark-submit command.\"\"\"\n\ndef launch(\n self,\n\ \ info: \"StepRunInfo\",\n entrypoint_command: List[str],\n\ \ ) -> None:\n \"\"\"Launches the step on Spark.\"\"\"\n\nUnder the\ \ base configuration, you will see the main configuration parameters:\n\nmaster\ \ is the master URL for the cluster where Spark will run. You might see different\ \ schemes for this URL with varying cluster managers such as Mesos, YARN, or Kubernetes.\n\ \ndeploy_mode can either be 'cluster' (default) or 'client' and it decides where\ \ the driver node of the application will run.\n\nsubmit_args is the JSON string\ \ of a dictionary, which will be used to define additional parameters if required\ \ ( Spark has a wide variety of parameters, thus including them all in a single\ \ class was deemed unnecessary.).\n\nIn addition to this configuration, the launch\ \ method of the step operator gets additional configuration parameters from the\ \ DockerSettings and ResourceSettings. As a result, the overall configuration\ \ happens in 4 base methods:\n\n_resource_configuration translates the ZenML ResourceSettings\ \ object to Spark's own resource configuration.\n\n_backend_configuration is responsible\ \ for cluster-manager-specific configuration." - source_sentence: What are the steps to configure RBAC for Spark in Kubernetes and register the stack component using ZenML? sentences: - " build to finish. More information: Build Timeout.We can register the image builder\ \ and use it in our active stack:\n\nzenml image-builder register \ \ \\\n --flavor=gcp \\\n --cloud_builder_image= \\\n\ \ --network= \\\n --build_timeout=\n\ \n# Register and activate a stack with the new image builder\nzenml stack register\ \ -i ... --set\n\nYou also need to set up authentication\ \ required to access the Cloud Build GCP services.\n\nAuthentication Methods\n\ \nIntegrating and using a GCP Image Builder in your pipelines is not possible\ \ without employing some form of authentication. If you're looking for a quick\ \ way to get started locally, you can use the Local Authentication method. However,\ \ the recommended way to authenticate to the GCP cloud platform is through a GCP\ \ Service Connector. This is particularly useful if you are configuring ZenML\ \ stacks that combine the GCP Image Builder with other remote stack components\ \ also running in GCP.\n\nThis method uses the implicit GCP authentication available\ \ in the environment where the ZenML code is running. On your local machine, this\ \ is the quickest way to configure a GCP Image Builder. You don't need to supply\ \ credentials explicitly when you register the GCP Image Builder, as it leverages\ \ the local credentials and configuration that the Google Cloud CLI stores on\ \ your local machine. However, you will need to install and set up the Google\ \ Cloud CLI on your machine as a prerequisite, as covered in the Google Cloud\ \ documentation , before you register the GCP Image Builder.\n\nStacks using the\ \ GCP Image Builder set up with local authentication are not portable across environments.\ \ To make ZenML pipelines fully portable, it is recommended to use a GCP Service\ \ Connector to authenticate your GCP Image Builder to the GCP cloud platform." - ' your GCP Image Builder to the GCP cloud platform.To set up the GCP Image Builder to authenticate to GCP and access the GCP Cloud Build services, it is recommended to leverage the many features provided by the GCP Service Connector such as auto-configuration, best security practices regarding long-lived credentials and reusing the same credentials across multiple stack components. If you don''t already have a GCP Service Connector configured in your ZenML deployment, you can register one using the interactive CLI command. You also have the option to configure a GCP Service Connector that can be used to access more than just the GCP Cloud Build service: zenml service-connector register --type gcp -i A non-interactive CLI example that leverages the Google Cloud CLI configuration on your local machine to auto-configure a GCP Service Connector for the GCP Cloud Build service: zenml service-connector register --type gcp --resource-type gcp-generic --resource-name --auto-configure Example Command Output $ zenml service-connector register gcp-generic --type gcp --resource-type gcp-generic --auto-configure Successfully registered service connector `gcp-generic` with access to the following resources: ┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓ ┃ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠────────────────┼────────────────┨ ┃ πŸ”΅ gcp-generic β”‚ zenml-core ┃ ┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ Note: Please remember to grant the entity associated with your GCP credentials permissions to access the Cloud Build API and to run Cloud Builder jobs (e.g. the Cloud Build Editor IAM role). The GCP Service Connector supports many different authentication methods with different levels of security and convenience. You should pick the one that best fits your use case. If you already have one or more GCP Service Connectors configured in your ZenML deployment, you can check which of them can be used to access generic GCP resources like the GCP Image Builder required for your GCP Image Builder by running e.g.:' - "ngs/python/Dockerfile -u 0 build\n\nConfiguring RBACAdditionally, you may need\ \ to create the several resources in Kubernetes in order to give Spark access\ \ to edit/manage your driver executor pods.\n\nTo do so, create a file called\ \ rbac.yaml with the following content:\n\napiVersion: v1\nkind: Namespace\nmetadata:\n\ \ name: spark-namespace\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n\ \ name: spark-service-account\n namespace: spark-namespace\n---\napiVersion:\ \ rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: spark-role\n\ \ namespace: spark-namespace\nsubjects:\n - kind: ServiceAccount\n name:\ \ spark-service-account\n namespace: spark-namespace\nroleRef:\n kind: ClusterRole\n\ \ name: edit\n apiGroup: rbac.authorization.k8s.io\n---\n\nAnd then execute\ \ the following command to create the resources:\n\naws eks --region=$REGION update-kubeconfig\ \ --name=$EKS_CLUSTER_NAME\n\nkubectl create -f rbac.yaml\n\nLastly, note down\ \ the namespace and the name of the service account since you will need them when\ \ registering the stack component in the next step.\n\nHow to use it\n\nTo use\ \ the KubernetesSparkStepOperator, you need:\n\nthe ZenML spark integration. If\ \ you haven't installed it already, run\n\nzenml integration install spark\n\n\ Docker installed and running.\n\nA remote artifact store as part of your stack.\n\ \nA remote container registry as part of your stack.\n\nA Kubernetes cluster deployed.\n\ \nWe can then register the step operator and use it in our active stack:\n\nzenml\ \ step-operator register spark_step_operator \\\n\t--flavor=spark-kubernetes \\\ \n\t--master=k8s://$EKS_API_SERVER_ENDPOINT \\\n\t--namespace=\ \ \\\n\t--service_account=\n\n# Register the\ \ stack\nzenml stack register spark_stack \\\n -o default \\\n -s spark_step_operator\ \ \\\n -a spark_artifact_store \\\n -c spark_container_registry \\\n \ \ -i local_builder \\\n --set" - source_sentence: Where can I find older versions of ZenML documentation? sentences: - 'ZenML - Bridging the gap between ML & Ops Legacy Docs Bleeding EdgeLegacy Docs0.67.0 πŸ§™β€β™‚οΈFind older version our docs Powered by GitBook' - 'ZenML - Bridging the gap between ML & Ops Legacy Docs Bleeding EdgeLegacy Docs0.67.0 πŸ§™β€β™‚οΈFind older version our docs Powered by GitBook' - "tainer_registry \\\n -i local_builder \\\n --setOnce you added the step\ \ operator to your active stack, you can use it to execute individual steps of\ \ your pipeline by specifying it in the @step decorator as follows:\n\nfrom zenml\ \ import step\n\n@step(step_operator=)\ndef step_on_spark(...)\ \ -> ...:\n \"\"\"Some step that should run with Spark on Kubernetes.\"\"\"\ \n ...\n\nAfter successfully running any step with a KubernetesSparkStepOperator,\ \ you should be able to see that a Spark driver pod was created in your cluster\ \ for each pipeline step when running kubectl get pods -n $KUBERNETES_NAMESPACE.\n\ \nInstead of hardcoding a step operator name, you can also use the Client to dynamically\ \ use the step operator of your active stack:\n\nfrom zenml.client import Client\n\ \nstep_operator = Client().active_stack.step_operator\n\n@step(step_operator=step_operator.name)\n\ def step_on_spark(...) -> ...:\n ...\n\nAdditional configuration\n\nFor additional\ \ configuration of the Spark step operator, you can pass SparkStepOperatorSettings\ \ when defining or running your pipeline. Check out the SDK docs for a full list\ \ of available attributes and this docs page for more information on how to specify\ \ settings.\n\nPreviousKubernetesNextDevelop a Custom Step Operator\n\nLast updated\ \ 4 months ago" pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: zenml/finetuned-snowflake-arctic-embed-m-v1.5 results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.75 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.75 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.75 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.9077324383928644 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.875 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.875 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.75 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.75 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.75 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.9077324383928644 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.875 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.875 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.75 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.75 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.75 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.9077324383928644 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.875 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.875 name: Cosine Map@100 --- # zenml/finetuned-snowflake-arctic-embed-m-v1.5 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Training Dataset:** - json - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the πŸ€— Hub model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5") # Run inference sentences = [ 'Where can I find older versions of ZenML documentation?', 'ZenML - Bridging the gap between ML & Ops\n\nLegacy Docs\n\nBleeding EdgeLegacy Docs0.67.0\n\nπŸ§™\u200d♂️Find older version our docs\n\nPowered by GitBook', 'tainer_registry \\\n -i local_builder \\\n --setOnce you added the step operator to your active stack, you can use it to execute individual steps of your pipeline by specifying it in the @step decorator as follows:\n\nfrom zenml import step\n\n@step(step_operator=)\ndef step_on_spark(...) -> ...:\n """Some step that should run with Spark on Kubernetes."""\n ...\n\nAfter successfully running any step with a KubernetesSparkStepOperator, you should be able to see that a Spark driver pod was created in your cluster for each pipeline step when running kubectl get pods -n $KUBERNETES_NAMESPACE.\n\nInstead of hardcoding a step operator name, you can also use the Client to dynamically use the step operator of your active stack:\n\nfrom zenml.client import Client\n\nstep_operator = Client().active_stack.step_operator\n\n@step(step_operator=step_operator.name)\ndef step_on_spark(...) -> ...:\n ...\n\nAdditional configuration\n\nFor additional configuration of the Spark step operator, you can pass SparkStepOperatorSettings when defining or running your pipeline. Check out the SDK docs for a full list of available attributes and this docs page for more information on how to specify settings.\n\nPreviousKubernetesNextDevelop a Custom Step Operator\n\nLast updated 4 months ago', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:--------| | cosine_accuracy@1 | 1.0 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 1.0 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 1.0 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 1.0 | | cosine_mrr@10 | 1.0 | | **cosine_map@100** | **1.0** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:----------| | cosine_accuracy@1 | 0.75 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 0.75 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 0.75 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 0.9077 | | cosine_mrr@10 | 0.875 | | **cosine_map@100** | **0.875** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:----------| | cosine_accuracy@1 | 0.75 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 0.75 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 0.75 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 0.9077 | | cosine_mrr@10 | 0.875 | | **cosine_map@100** | **0.875** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:----------| | cosine_accuracy@1 | 0.75 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 0.75 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 0.75 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 0.9077 | | cosine_mrr@10 | 0.875 | | **cosine_map@100** | **0.875** | ## Training Details ### Training Dataset #### json * Dataset: json * Size: 36 training samples * Columns: positive and anchor * Approximate statistics based on the first 36 samples: | | positive | anchor | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 13 tokens
  • mean: 22.58 tokens
  • max: 44 tokens
|
  • min: 32 tokens
  • mean: 300.72 tokens
  • max: 512 tokens
| * Samples: | positive | anchor | |:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | How do you configure ZenML to display data visualizations in the dashboard? | πŸ“ŠVisualizing artifacts

Configuring ZenML to display data visualizations in the dashboard.

PreviousRegister Existing Data as a ZenML ArtifactNextDefault visualizations

Last updated 4 months ago | | How does the model deployer in ZenML facilitate the deployment and management of machine learning models? | Develop a Custom Model Deployer

Learning how to develop a custom model deployer.

Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML's component flavor concepts.

To deploy and manage your trained machine-learning models, ZenML provides a stack component called Model Deployer. This component is responsible for interacting with the deployment tool, framework, or platform.

When present in a stack, the model deployer can also act as a registry for models that are served with ZenML. You can use the model deployer to list all models that are currently deployed for online inference or filtered according to a particular pipeline run or step, or to suspend, resume or delete an external model server managed through ZenML.

Base Abstraction

In ZenML, the base abstraction of the model deployer is built on top of three major criteria:

It needs to ensure efficient deployment and management of models in accordance with the specific requirements of the serving infrastructure, by holding all the stack-related configuration attributes required to interact with the remote model serving tool, service, or platform.

It needs to implement the continuous deployment logic necessary to deploy models in a way that updates an existing model server that is already serving a previous version of the same model instead of creating a new model server for every new model version (see the deploy_model abstract method). This functionality can be consumed directly from ZenML pipeline steps, but it can also be used outside the pipeline to deploy ad-hoc models. It is also usually coupled with a standard model deployer step, implemented by each integration, that hides the details of the deployment process from the user. | | How can I track the improvement of my RAG pipeline using evaluation and metrics? | Evaluation and metrics

Track how your RAG pipeline improves using evaluation and metrics.

PreviousBasic RAG inference pipelineNextEvaluation in 65 lines of code

Last updated 4 months ago | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `tf32`: False - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: False - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_384_cosine_map@100 | dim_256_cosine_map@100 | dim_128_cosine_map@100 | dim_64_cosine_map@100 | |:-------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | **1.0** | **1** | **0.875** | **0.875** | **0.875** | **0.875** | | 2.0 | 3 | 1.0 | 0.875 | 0.875 | 0.875 | | 3.0 | 4 | 1.0 | 0.875 | 0.875 | 0.875 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.11.10 - Sentence Transformers: 3.2.1 - Transformers: 4.43.1 - PyTorch: 2.5.1+cu124 - Accelerate: 1.1.0 - Datasets: 3.1.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```