--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:36 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: Snowflake/snowflake-arctic-embed-m-v1.5 widget: - source_sentence: How do you configure the necessary RBAC resources in Kubernetes to enable Spark access for managing driver executor pods, and what are the subsequent steps needed to register the stack component using ZenML? sentences: - 'Google Cloud Image Builder Building container images with Google Cloud Build The Google Cloud image builder is an image builder flavor provided by the ZenML gcp integration that uses Google Cloud Build to build container images. When to use it You should use the Google Cloud image builder if: you''re unable to install or use Docker on your client machine. you''re already using GCP. your stack is mainly composed of other Google Cloud components such as the GCS Artifact Store or the Vertex Orchestrator. How to deploy it Would you like to skip ahead and deploy a full ZenML cloud stack already, including the Google Cloud image builder? Check out the in-browser stack deployment wizard, the stack registration wizard, or the ZenML GCP Terraform module for a shortcut on how to deploy & register this stack component. In order to use the ZenML Google Cloud image builder you need to enable Google Cloud Build relevant APIs on the Google Cloud project. How to use it To use the Google Cloud image builder, we need: The ZenML gcp integration installed. If you haven''t done so, run: zenml integration install gcp A GCP Artifact Store where the build context will be uploaded, so Google Cloud Build can access it. A GCP container registry where the built image will be pushed. Optionally, the GCP project ID in which you want to run the build and a service account with the needed permissions to run the build. If not provided, then the project ID and credentials will be inferred from the environment. Optionally, you can change: the Docker image used by Google Cloud Build to execute the steps to build and push the Docker image. By default, the builder image will be ''gcr.io/cloud-builders/docker''. The network to which the container used to build the ZenML pipeline Docker image will be attached. More information: Cloud build network. The build timeout for the build, and for the blocking operation waiting for the build to finish. More information: Build Timeout.' - "_run.steps[step_name]\n whylogs_step.visualize()if __name__ == \"__main__\"\ :\n visualize_statistics(\"data_loader\")\n visualize_statistics(\"train_data_profiler\"\ , \"test_data_profiler\")\n\nPreviousEvidentlyNextDevelop a custom data validator\n\ \nLast updated 1 month ago" - "ngs/python/Dockerfile -u 0 build\n\nConfiguring RBACAdditionally, you may need\ \ to create the several resources in Kubernetes in order to give Spark access\ \ to edit/manage your driver executor pods.\n\nTo do so, create a file called\ \ rbac.yaml with the following content:\n\napiVersion: v1\nkind: Namespace\nmetadata:\n\ \ name: spark-namespace\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n\ \ name: spark-service-account\n namespace: spark-namespace\n---\napiVersion:\ \ rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: spark-role\n\ \ namespace: spark-namespace\nsubjects:\n - kind: ServiceAccount\n name:\ \ spark-service-account\n namespace: spark-namespace\nroleRef:\n kind: ClusterRole\n\ \ name: edit\n apiGroup: rbac.authorization.k8s.io\n---\n\nAnd then execute\ \ the following command to create the resources:\n\naws eks --region=$REGION update-kubeconfig\ \ --name=$EKS_CLUSTER_NAME\n\nkubectl create -f rbac.yaml\n\nLastly, note down\ \ the namespace and the name of the service account since you will need them when\ \ registering the stack component in the next step.\n\nHow to use it\n\nTo use\ \ the KubernetesSparkStepOperator, you need:\n\nthe ZenML spark integration. If\ \ you haven't installed it already, run\n\nzenml integration install spark\n\n\ Docker installed and running.\n\nA remote artifact store as part of your stack.\n\ \nA remote container registry as part of your stack.\n\nA Kubernetes cluster deployed.\n\ \nWe can then register the step operator and use it in our active stack:\n\nzenml\ \ step-operator register spark_step_operator \\\n\t--flavor=spark-kubernetes \\\ \n\t--master=k8s://$EKS_API_SERVER_ENDPOINT \\\n\t--namespace=\ \ \\\n\t--service_account=\n\n# Register the\ \ stack\nzenml stack register spark_stack \\\n -o default \\\n -s spark_step_operator\ \ \\\n -a spark_artifact_store \\\n -c spark_container_registry \\\n \ \ -i local_builder \\\n --set" - source_sentence: What is the function of a ZenML BaseService registry in the context of model deployment? sentences: - "\U0001F5C4️Handle Data/Artifacts\n\nStep outputs in ZenML are stored in the artifact\ \ store. This enables caching, lineage and auditability. Using type annotations\ \ helps with transparency, passing data between steps, and serializing/des\n\n\ For best results, use type annotations for your outputs. This is good coding practice\ \ for transparency, helps ZenML handle passing data between steps, and also enables\ \ ZenML to serialize and deserialize (referred to as 'materialize' in ZenML) the\ \ data.\n\n@step\ndef load_data(parameter: int) -> Dict[str, Any]:\n\n# do something\ \ with the parameter here\n\ntraining_data = [[1, 2], [3, 4], [5, 6]]\n labels\ \ = [0, 1, 0]\n return {'features': training_data, 'labels': labels}\n\n@step\n\ def train_model(data: Dict[str, Any]) -> None:\n total_features = sum(map(sum,\ \ data['features']))\n total_labels = sum(data['labels'])\n \n # Train\ \ some model here\n \n print(f\"Trained model using {len(data['features'])}\ \ data points. \"\n f\"Feature sum is {total_features}, label sum is\ \ {total_labels}\")\n\n@pipeline \ndef simple_ml_pipeline(parameter: int):\n\ \ dataset = load_data(parameter=parameter) # Get the output \n train_model(dataset)\ \ # Pipe the previous step output into the downstream step\n\nIn this code, we\ \ define two steps: load_data and train_model. The load_data step takes an integer\ \ parameter and returns a dictionary containing training data and labels. The\ \ train_model step receives the dictionary from load_data, extracts the features\ \ and labels, and trains a model (not shown here).\n\nFinally, we define a pipeline\ \ simple_ml_pipeline that chains the load_data and train_model steps together.\ \ The output from load_data is passed as input to train_model, demonstrating how\ \ data flows between steps in a ZenML pipeline.\n\nPreviousDisable colorful loggingNextHow\ \ ZenML stores data\n\nLast updated 4 months ago" - 'πŸ§™Installation Installing ZenML and getting started. ZenML is a Python package that can be installed directly via pip: pip install zenml Note that ZenML currently supports Python 3.8, 3.9, 3.10, and 3.11. Please make sure that you are using a supported Python version. Install with the dashboard ZenML comes bundled with a web dashboard that lives inside a sister repository. In order to get access to the dashboard locally, you need to launch the ZenML Server and Dashboard locally. For this, you need to install the optional dependencies for the ZenML Server: pip install "zenml[server]" We highly encourage you to install ZenML in a virtual environment. At ZenML, We like to use virtualenvwrapper or pyenv-virtualenv to manage our Python virtual environments. Installing onto MacOS with Apple Silicon (M1, M2) A change in how forking works on Macs running on Apple Silicon means that you should set the following environment variable which will ensure that your connections to the server remain unbroken: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES You can read more about this here. This environment variable is needed if you are working with a local server on your Mac, but if you''re just using ZenML as a client / CLI and connecting to a deployed server then you don''t need to set it. Nightly builds ZenML also publishes nightly builds under the zenml-nightly package name. These are built from the latest develop branch (to which work ready for release is published) and are not guaranteed to be stable. To install the nightly build, run: pip install zenml-nightly Verifying installations Once the installation is completed, you can check whether the installation was successful either through Bash: zenml version or through Python: import zenml print(zenml.__version__) If you would like to learn more about the current release, please visit our PyPi package page. Running with Docker' - "e details of the deployment process from the user.It needs to act as a ZenML\ \ BaseService registry, where every BaseService instance is used as an internal\ \ representation of a remote model server (see the find_model_server abstract\ \ method). To achieve this, it must be able to re-create the configuration of\ \ a BaseService from information that is persisted externally, alongside, or even\ \ as part of the remote model server configuration itself. For example, for model\ \ servers that are implemented as Kubernetes resources, the BaseService instances\ \ can be serialized and saved as Kubernetes resource annotations. This allows\ \ the model deployer to keep track of all externally running model servers and\ \ to re-create their corresponding BaseService instance representations at any\ \ given time. The model deployer also defines methods that implement basic life-cycle\ \ management on remote model servers outside the coverage of a pipeline (see stop_model_server\ \ , start_model_server and delete_model_server).\n\nPutting all these considerations\ \ together, we end up with the following interface:\n\nfrom abc import ABC, abstractmethod\n\ from typing import Dict, List, Optional, Type\nfrom uuid import UUID\n\nfrom zenml.enums\ \ import StackComponentType\nfrom zenml.services import BaseService, ServiceConfig\n\ from zenml.stack import StackComponent, StackComponentConfig, Flavor\n\nDEFAULT_DEPLOYMENT_START_STOP_TIMEOUT\ \ = 300\n\nclass BaseModelDeployerConfig(StackComponentConfig):\n \"\"\"Base\ \ class for all ZenML model deployer configurations.\"\"\"\n\nclass BaseModelDeployer(StackComponent,\ \ ABC):\n \"\"\"Base class for all ZenML model deployers.\"\"\"\n\n@abstractmethod\n\ \ def perform_deploy_model(\n self,\n id: UUID,\n config:\ \ ServiceConfig,\n timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n\ \ ) -> BaseService:\n \"\"\"Abstract method to deploy a model.\"\"\"" - source_sentence: How can I implement the abstract method to deploy a model using ZenML? sentences: - "> \\\n --build_timeout=# Register and set a stack\ \ with the new image builder\nzenml stack register -i \ \ ... --set\n\nCaveats\n\nAs described in this Google Cloud Build documentation\ \ page, Google Cloud Build uses containers to execute the build steps which are\ \ automatically attached to a network called cloudbuild that provides some Application\ \ Default Credentials (ADC), that allow the container to be authenticated and\ \ therefore use other GCP services.\n\nBy default, the GCP Image Builder is executing\ \ the build command of the ZenML Pipeline Docker image with the option --network=cloudbuild,\ \ so the ADC provided by the cloudbuild network can also be used in the build.\ \ This is useful if you want to install a private dependency from a GCP Artifact\ \ Registry, but you will also need to use a custom base parent image with the\ \ keyrings.google-artifactregistry-auth installed, so pip can connect and authenticate\ \ in the private artifact registry to download the dependency.\n\nFROM zenmldocker/zenml:latest\n\ \nRUN pip install keyrings.google-artifactregistry-auth\n\nThe above Dockerfile\ \ uses zenmldocker/zenml:latest as a base image, but is recommended to change\ \ the tag to specify the ZenML version and Python version like 0.33.0-py3.10.\n\ \nPreviousKaniko Image BuilderNextDevelop a Custom Image Builder\n\nLast updated\ \ 21 days ago" - ":\n \"\"\"Abstract method to deploy a model.\"\"\"@staticmethod\n @abstractmethod\n\ \ def get_model_server_info(\n service: BaseService,\n ) -> Dict[str,\ \ Optional[str]]:\n \"\"\"Give implementation-specific way to extract relevant\ \ model server\n properties for the user.\"\"\"\n\n@abstractmethod\n \ \ def perform_stop_model(\n self,\n service: BaseService,\n \ \ timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n force: bool\ \ = False,\n ) -> BaseService:\n \"\"\"Abstract method to stop a model\ \ server.\"\"\"\n\n@abstractmethod\n def perform_start_model(\n self,\n\ \ service: BaseService,\n timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n\ \ ) -> BaseService:\n \"\"\"Abstract method to start a model server.\"\ \"\"\n\n@abstractmethod\n def perform_delete_model(\n self,\n \ \ service: BaseService,\n timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n\ \ force: bool = False,\n ) -> None:\n \"\"\"Abstract method to\ \ delete a model server.\"\"\"\n\nclass BaseModelDeployerFlavor(Flavor):\n \ \ \"\"\"Base class for model deployer flavors.\"\"\"\n\n@property\n @abstractmethod\n\ \ def name(self):\n \"\"\"Returns the name of the flavor.\"\"\"\n\n\ @property\n def type(self) -> StackComponentType:\n \"\"\"Returns the\ \ flavor type.\n\nReturns:\n The flavor type.\n \"\"\"\n \ \ return StackComponentType.MODEL_DEPLOYER\n\n@property\n def config_class(self)\ \ -> Type[BaseModelDeployerConfig]:\n \"\"\"Returns `BaseModelDeployerConfig`\ \ config class.\n\nReturns:\n The config class.\n \"\"\"\ \n return BaseModelDeployerConfig\n\n@property\n @abstractmethod\n \ \ def implementation_class(self) -> Type[BaseModelDeployer]:\n \"\"\"\ The class that implements the model deployer.\"\"\"\n\nThis is a slimmed-down\ \ version of the base implementation which aims to highlight the abstraction layer.\ \ In order to see the full implementation and get the complete docstrings, please\ \ check the SDK docs .\n\nBuilding your own model deployers" - "se you decide to switch to another Data Validator.All you have to do is call\ \ the whylogs Data Validator methods when you need to interact with whylogs to\ \ generate data profiles. You may optionally enable whylabs logging to automatically\ \ upload the returned whylogs profile to WhyLabs, e.g.:\n\nimport pandas as pd\n\ from whylogs.core import DatasetProfileView\nfrom zenml.integrations.whylogs.data_validators.whylogs_data_validator\ \ import (\n WhylogsDataValidator,\n)\nfrom zenml.integrations.whylogs.flavors.whylogs_data_validator_flavor\ \ import (\n WhylogsDataValidatorSettings,\n)\nfrom zenml import step\n\nwhylogs_settings\ \ = WhylogsDataValidatorSettings(\n enable_whylabs=True, dataset_id=\"\"\ \n)\n\n@step(\n settings={\n \"data_validator\": whylogs_settings\n\ \ }\n)\ndef data_profiler(\n dataset: pd.DataFrame,\n) -> DatasetProfileView:\n\ \ \"\"\"Custom data profiler step with whylogs\n\nArgs:\n dataset: a\ \ Pandas DataFrame\n\nReturns:\n Whylogs profile generated for the data\n\ \ \"\"\"\n\n# validation pre-processing (e.g. dataset preparation) can take\ \ place here\n\ndata_validator = WhylogsDataValidator.get_active_data_validator()\n\ \ profile = data_validator.data_profiling(\n dataset,\n )\n #\ \ optionally upload the profile to WhyLabs, if WhyLabs credentials are configured\n\ \ data_validator.upload_profile_view(profile)\n\n# validation post-processing\ \ (e.g. interpret results, take actions) can happen here\n\nreturn profile\n\n\ Have a look at the complete list of methods and parameters available in the WhylogsDataValidator\ \ API in the SDK docs.\n\nCall whylogs directly\n\nYou can use the whylogs library\ \ directly in your custom pipeline steps, and only leverage ZenML's capability\ \ of serializing, versioning and storing the DatasetProfileView objects in its\ \ Artifact Store. You may optionally enable whylabs logging to automatically upload\ \ the returned whylogs profile to WhyLabs, e.g.:" - source_sentence: How can I register and configure a GCP Service Connector for accessing GCP Cloud Build services in ZenML? sentences: - 'System Architectures Different variations of the ZenML architecture depending on your needs. PreviousZenML ProNextZenML SaaS Last updated 21 days ago' - "quired for your GCP Image Builder by running e.g.:zenml service-connector list-resources\ \ --resource-type gcp-generic\n\nExample Command Output\n\nThe following 'gcp-generic'\ \ resources can be accessed by service connectors that you have configured:\n\ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓\n\ ┃ CONNECTOR ID β”‚ CONNECTOR NAME β”‚ CONNECTOR TYPE β”‚ RESOURCE\ \ TYPE β”‚ RESOURCE NAMES ┃\n┠──────────────────────────────────────┼────────────────┼────────────────┼────────────────┼────────────────┨\n\ ┃ bfdb657d-d808-47e7-9974-9ba6e4919d83 β”‚ gcp-generic β”‚ \U0001F535 gcp \ \ β”‚ \U0001F535 gcp-generic β”‚ zenml-core ┃\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛\n\ \nAfter having set up or decided on a GCP Service Connector to use to authenticate\ \ to GCP, you can register the GCP Image Builder as follows:\n\nzenml image-builder\ \ register \\\n --flavor=gcp \\\n --cloud_builder_image=\ \ \\\n --network= \\\n --build_timeout=\n\ \n# Connect the GCP Image Builder to GCP via a GCP Service Connector\nzenml image-builder\ \ connect -i\n\nA non-interactive version that connects the\ \ GCP Image Builder to a target GCP Service Connector:\n\nzenml image-builder\ \ connect --connector \n\nExample Command Output" - ' your GCP Image Builder to the GCP cloud platform.To set up the GCP Image Builder to authenticate to GCP and access the GCP Cloud Build services, it is recommended to leverage the many features provided by the GCP Service Connector such as auto-configuration, best security practices regarding long-lived credentials and reusing the same credentials across multiple stack components. If you don''t already have a GCP Service Connector configured in your ZenML deployment, you can register one using the interactive CLI command. You also have the option to configure a GCP Service Connector that can be used to access more than just the GCP Cloud Build service: zenml service-connector register --type gcp -i A non-interactive CLI example that leverages the Google Cloud CLI configuration on your local machine to auto-configure a GCP Service Connector for the GCP Cloud Build service: zenml service-connector register --type gcp --resource-type gcp-generic --resource-name --auto-configure Example Command Output $ zenml service-connector register gcp-generic --type gcp --resource-type gcp-generic --auto-configure Successfully registered service connector `gcp-generic` with access to the following resources: ┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓ ┃ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠────────────────┼────────────────┨ ┃ πŸ”΅ gcp-generic β”‚ zenml-core ┃ ┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ Note: Please remember to grant the entity associated with your GCP credentials permissions to access the Cloud Build API and to run Cloud Builder jobs (e.g. the Cloud Build Editor IAM role). The GCP Service Connector supports many different authentication methods with different levels of security and convenience. You should pick the one that best fits your use case. If you already have one or more GCP Service Connectors configured in your ZenML deployment, you can check which of them can be used to access generic GCP resources like the GCP Image Builder required for your GCP Image Builder by running e.g.:' - source_sentence: How can ZenML be used to finetune LLMs for specific tasks or to improve their performance and cost? sentences: - " build to finish. More information: Build Timeout.We can register the image builder\ \ and use it in our active stack:\n\nzenml image-builder register \ \ \\\n --flavor=gcp \\\n --cloud_builder_image= \\\n\ \ --network= \\\n --build_timeout=\n\ \n# Register and activate a stack with the new image builder\nzenml stack register\ \ -i ... --set\n\nYou also need to set up authentication\ \ required to access the Cloud Build GCP services.\n\nAuthentication Methods\n\ \nIntegrating and using a GCP Image Builder in your pipelines is not possible\ \ without employing some form of authentication. If you're looking for a quick\ \ way to get started locally, you can use the Local Authentication method. However,\ \ the recommended way to authenticate to the GCP cloud platform is through a GCP\ \ Service Connector. This is particularly useful if you are configuring ZenML\ \ stacks that combine the GCP Image Builder with other remote stack components\ \ also running in GCP.\n\nThis method uses the implicit GCP authentication available\ \ in the environment where the ZenML code is running. On your local machine, this\ \ is the quickest way to configure a GCP Image Builder. You don't need to supply\ \ credentials explicitly when you register the GCP Image Builder, as it leverages\ \ the local credentials and configuration that the Google Cloud CLI stores on\ \ your local machine. However, you will need to install and set up the Google\ \ Cloud CLI on your machine as a prerequisite, as covered in the Google Cloud\ \ documentation , before you register the GCP Image Builder.\n\nStacks using the\ \ GCP Image Builder set up with local authentication are not portable across environments.\ \ To make ZenML pipelines fully portable, it is recommended to use a GCP Service\ \ Connector to authenticate your GCP Image Builder to the GCP cloud platform." - 'Finetuning LLMs with ZenML Finetune LLMs for specific tasks or to improve performance and cost. PreviousEvaluating finetuned embeddingsNextSet up a project repository Last updated 6 months ago' - "Spark\n\nExecuting individual steps on Spark\n\nThe spark integration brings\ \ two different step operators:\n\nStep Operator: The SparkStepOperator serves\ \ as the base class for all the Spark-related step operators.\n\nStep Operator:\ \ The KubernetesSparkStepOperator is responsible for launching ZenML steps as\ \ Spark applications with Kubernetes as a cluster manager.\n\nStep Operators:\ \ SparkStepOperator\n\nA summarized version of the implementation can be summarized\ \ in two parts. First, the configuration:\n\nfrom typing import Optional, Dict,\ \ Any\nfrom zenml.step_operators import BaseStepOperatorConfig\n\nclass SparkStepOperatorConfig(BaseStepOperatorConfig):\n\ \ \"\"\"Spark step operator config.\n\nAttributes:\n master: is the\ \ master URL for the cluster. You might see different\n schemes for\ \ different cluster managers which are supported by Spark\n like Mesos,\ \ YARN, or Kubernetes. Within the context of this PR,\n the implementation\ \ supports Kubernetes as a cluster manager.\n deploy_mode: can either be\ \ 'cluster' (default) or 'client' and it\n decides where the driver\ \ node of the application will run.\n submit_kwargs: is the JSON string\ \ of a dict, which will be used\n to define additional params if required\ \ (Spark has quite a\n lot of different parameters, so including them,\ \ all in the step\n operator was not implemented).\n \"\"\"\n\n\ master: str\n deploy_mode: str = \"cluster\"\n submit_kwargs: Optional[Dict[str,\ \ Any]] = None\n\nand then the implementation:\n\nfrom typing import List\nfrom\ \ pyspark.conf import SparkConf\n\nfrom zenml.step_operators import BaseStepOperator\n\ \nclass SparkStepOperator(BaseStepOperator):\n \"\"\"Base class for all Spark-related\ \ step operators.\"\"\"\n\ndef _resource_configuration(\n self,\n \ \ spark_config: SparkConf,\n resource_configuration: \"ResourceSettings\"\ ,\n ) -> None:\n \"\"\"Configures Spark to handle the resource configuration.\"\ \"\"" pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: zenml/finetuned-snowflake-arctic-embed-m-v1.5 results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 1.0 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 1.0 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 1.0 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 1.0 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 1.0 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.2 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.1 name: Cosine Precision@10 - type: cosine_recall@1 value: 1.0 name: Cosine Recall@1 - type: cosine_recall@3 value: 1.0 name: Cosine Recall@3 - type: cosine_recall@5 value: 1.0 name: Cosine Recall@5 - type: cosine_recall@10 value: 1.0 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 1.0 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 1.0 name: Cosine Mrr@10 - type: cosine_map@100 value: 1.0 name: Cosine Map@100 --- # zenml/finetuned-snowflake-arctic-embed-m-v1.5 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-m-v1.5](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Training Dataset:** - json - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the πŸ€— Hub model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5") # Run inference sentences = [ 'How can ZenML be used to finetune LLMs for specific tasks or to improve their performance and cost?', 'Finetuning LLMs with ZenML\n\nFinetune LLMs for specific tasks or to improve performance and cost.\n\nPreviousEvaluating finetuned embeddingsNextSet up a project repository\n\nLast updated 6 months ago', 'Spark\n\nExecuting individual steps on Spark\n\nThe spark integration brings two different step operators:\n\nStep Operator: The SparkStepOperator serves as the base class for all the Spark-related step operators.\n\nStep Operator: The KubernetesSparkStepOperator is responsible for launching ZenML steps as Spark applications with Kubernetes as a cluster manager.\n\nStep Operators: SparkStepOperator\n\nA summarized version of the implementation can be summarized in two parts. First, the configuration:\n\nfrom typing import Optional, Dict, Any\nfrom zenml.step_operators import BaseStepOperatorConfig\n\nclass SparkStepOperatorConfig(BaseStepOperatorConfig):\n """Spark step operator config.\n\nAttributes:\n master: is the master URL for the cluster. You might see different\n schemes for different cluster managers which are supported by Spark\n like Mesos, YARN, or Kubernetes. Within the context of this PR,\n the implementation supports Kubernetes as a cluster manager.\n deploy_mode: can either be \'cluster\' (default) or \'client\' and it\n decides where the driver node of the application will run.\n submit_kwargs: is the JSON string of a dict, which will be used\n to define additional params if required (Spark has quite a\n lot of different parameters, so including them, all in the step\n operator was not implemented).\n """\n\nmaster: str\n deploy_mode: str = "cluster"\n submit_kwargs: Optional[Dict[str, Any]] = None\n\nand then the implementation:\n\nfrom typing import List\nfrom pyspark.conf import SparkConf\n\nfrom zenml.step_operators import BaseStepOperator\n\nclass SparkStepOperator(BaseStepOperator):\n """Base class for all Spark-related step operators."""\n\ndef _resource_configuration(\n self,\n spark_config: SparkConf,\n resource_configuration: "ResourceSettings",\n ) -> None:\n """Configures Spark to handle the resource configuration."""', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:--------| | cosine_accuracy@1 | 1.0 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 1.0 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 1.0 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 1.0 | | cosine_mrr@10 | 1.0 | | **cosine_map@100** | **1.0** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:--------| | cosine_accuracy@1 | 1.0 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 1.0 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 1.0 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 1.0 | | cosine_mrr@10 | 1.0 | | **cosine_map@100** | **1.0** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:--------| | cosine_accuracy@1 | 1.0 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 1.0 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 1.0 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 1.0 | | cosine_mrr@10 | 1.0 | | **cosine_map@100** | **1.0** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:--------| | cosine_accuracy@1 | 1.0 | | cosine_accuracy@3 | 1.0 | | cosine_accuracy@5 | 1.0 | | cosine_accuracy@10 | 1.0 | | cosine_precision@1 | 1.0 | | cosine_precision@3 | 0.3333 | | cosine_precision@5 | 0.2 | | cosine_precision@10 | 0.1 | | cosine_recall@1 | 1.0 | | cosine_recall@3 | 1.0 | | cosine_recall@5 | 1.0 | | cosine_recall@10 | 1.0 | | cosine_ndcg@10 | 1.0 | | cosine_mrr@10 | 1.0 | | **cosine_map@100** | **1.0** | ## Training Details ### Training Dataset #### json * Dataset: json * Size: 36 training samples * Columns: positive and anchor * Approximate statistics based on the first 36 samples: | | positive | anchor | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 13 tokens
  • mean: 23.19 tokens
  • max: 48 tokens
|
  • min: 31 tokens
  • mean: 320.53 tokens
  • max: 512 tokens
| * Samples: | positive | anchor | |:------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Where can I find older versions of the ZenML documentation? | ZenML - Bridging the gap between ML & Ops

Legacy Docs

Bleeding EdgeLegacy Docs0.67.0

πŸ§™β€β™‚οΈFind older version our docs

Powered by GitBook | | How can I set up authentication for a GCP Image Builder when registering it in ZenML? | build to finish. More information: Build Timeout.We can register the image builder and use it in our active stack:

zenml image-builder register \
--flavor=gcp \
--cloud_builder_image= \
--network= \
--build_timeout=

# Register and activate a stack with the new image builder
zenml stack register -i ... --set

You also need to set up authentication required to access the Cloud Build GCP services.

Authentication Methods

Integrating and using a GCP Image Builder in your pipelines is not possible without employing some form of authentication. If you're looking for a quick way to get started locally, you can use the Local Authentication method. However, the recommended way to authenticate to the GCP cloud platform is through a GCP Service Connector. This is particularly useful if you are configuring ZenML stacks that combine the GCP Image Builder with other remote stack components also running in GCP.

This method uses the implicit GCP authentication available in the environment where the ZenML code is running. On your local machine, this is the quickest way to configure a GCP Image Builder. You don't need to supply credentials explicitly when you register the GCP Image Builder, as it leverages the local credentials and configuration that the Google Cloud CLI stores on your local machine. However, you will need to install and set up the Google Cloud CLI on your machine as a prerequisite, as covered in the Google Cloud documentation , before you register the GCP Image Builder.

Stacks using the GCP Image Builder set up with local authentication are not portable across environments. To make ZenML pipelines fully portable, it is recommended to use a GCP Service Connector to authenticate your GCP Image Builder to the GCP cloud platform. | | How can I switch to another Data Validator and enable WhyLabs logging for automatic profile uploads using ZenML? | se you decide to switch to another Data Validator.All you have to do is call the whylogs Data Validator methods when you need to interact with whylogs to generate data profiles. You may optionally enable whylabs logging to automatically upload the returned whylogs profile to WhyLabs, e.g.:

import pandas as pd
from whylogs.core import DatasetProfileView
from zenml.integrations.whylogs.data_validators.whylogs_data_validator import (
WhylogsDataValidator,
)
from zenml.integrations.whylogs.flavors.whylogs_data_validator_flavor import (
WhylogsDataValidatorSettings,
)
from zenml import step

whylogs_settings = WhylogsDataValidatorSettings(
enable_whylabs=True, dataset_id=""
)

@step(
settings={
"data_validator": whylogs_settings
}
)
def data_profiler(
dataset: pd.DataFrame,
) -> DatasetProfileView:
"""Custom data profiler step with whylogs

Args:
dataset: a Pandas DataFrame

Returns:
Whylogs profile generated for the data
"""

# validation pre-processing (e.g. dataset preparation) can take place here

data_validator = WhylogsDataValidator.get_active_data_validator()
profile = data_validator.data_profiling(
dataset,
)
# optionally upload the profile to WhyLabs, if WhyLabs credentials are configured
data_validator.upload_profile_view(profile)

# validation post-processing (e.g. interpret results, take actions) can happen here

return profile

Have a look at the complete list of methods and parameters available in the WhylogsDataValidator API in the SDK docs.

Call whylogs directly

You can use the whylogs library directly in your custom pipeline steps, and only leverage ZenML's capability of serializing, versioning and storing the DatasetProfileView objects in its Artifact Store. You may optionally enable whylabs logging to automatically upload the returned whylogs profile to WhyLabs, e.g.: | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `tf32`: False - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: False - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_384_cosine_map@100 | dim_256_cosine_map@100 | dim_128_cosine_map@100 | dim_64_cosine_map@100 | |:-------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | **1.0** | **1** | **1.0** | **1.0** | **1.0** | **1.0** | | 2.0 | 3 | 1.0 | 1.0 | 1.0 | 1.0 | | 3.0 | 4 | 1.0 | 1.0 | 1.0 | 1.0 | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.11.10 - Sentence Transformers: 3.2.1 - Transformers: 4.43.1 - PyTorch: 2.5.1+cu124 - Accelerate: 1.0.1 - Datasets: 3.0.2 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```