Skip to content

How to Build your own Helmholtz Cloud Agent

Here, we guide you through the first steps of writing and connecting your Helmholtz Cloud Agent (HCA) to the central Helmholtz Cloud resource management. A Helmholtz Cloud Agent is a small Python tool that runs close to a service and communicates with the Helmholtz Cloud. It receives resource requests and sends back information about the allocated resource. The communication with Helmholtz Cloud is handled automatically by a provided library so that you can focus on integrating your service.

Developing your first Helmholtz Cloud Agent

We provide a template repository that you can use as a starting point for your development. It contains a devcontainer setup with a local mock cloud. It also includes a sample Dockerfile that can be used to deploy your Helmholtz Cloud Agent.

Fork the template repository to start developing your own agent. If you are using Visual Studio Code with devcontainers you can simply check out the repository and open it. It will ask you to “Reopen in Container” which will launch the development environment. If you are not using VSCode you can just manually start the environment with the provided Docker Compose file in the dev folder.

Communicating with the Mock Cloud

The template comes with a sample Helmholtz Cloud Agent implementation that we will discuss later. For now, we will just start the mock Helmholtz Cloud and the Helmholtz Cloud Agent and test if everything is set up correctly.

To start the mock cloud run this in a shell in the hca_dev container: 

1
uv run fastapi dev ./dev/mock_portal.py

Open another separate shell and boot your Helmholtz Cloud Agent:

1
uv run python src/example.py

This will start the Helmholtz Cloud Agent and connect. The mock cloud comes with an API endpoint that you can access at http://localhost:8000/. The exact URL may be different on your machine - check the Ports tab in VSCode to access the page. Open the page to see the current contents of the two queues representing the directions from cloud to service and service to cloud. Until you have sent a message on each queue the response on that page may be empty. The plugin-template repository comes with a script that mocks the interaction of Helmholtz Cloud with the Helmholtz Cloud Agent.

To test the setup, we can start by sending a simple PingV1 message. Helmholtz Cloud may send this message at any time to check whether the Helmholtz Cloud Agent is still running. Your Helmholtz Cloud Agent will then automatically send a PingV1 response back to Helmholtz Cloud. While keeping the Helmholtz Cloud Agent and the mock cloud running, open a third shell and run:

1
uv run python dev/mock_portal.py send-ping

You should see an output with the message “still alive”. If yes, your development environment is set up correctly, and we can now proceed with resource requests. You should also see the queues appear at http://localhost:8000/. Please note the queues are flushed every time you restart the mock server.

Send your first Resource Request

In the same way that we sent the PingV1 message we can also send resource requests to the Helmholtz Cloud Agent. The message type is ResourceAllocateV1 and the actual content depends on your resource type and your policy definition in Plony as described in integration guide. However, the main structure is always the same three fields:

  • type: A string representing the title of the resource type JSON schema.
  • target_entity: A dictionary containing either the group URN or the user ID, depending on the selected policy.
  • specification: A dictionary containing the resource specification as defined in the JSON Schema properties of the resource type.

Using the Script

The sample Helmholtz Cloud Agent in the template accepts two different resource types (ChatTeamSpecV1 and ComputeResourceSpecV1). For testing, it randomly returns a ResourceCreatedV1 or ErrorV1 message without actually doing anything. Here, we will start by sending a ChatTeamSpecV1 to our Helmholtz Cloud Agent using the mock_portal.py scripts:

1
uv run python dev/mock_portal.py send-resource-allocate payloads/ChatTeamSpecV1.json

The mock_portal.py expects a JSON file with a payload that it will send to the Helmholtz Cloud Agent. Two sample payloads are already provided in the payloads folder. You can adapt and add payloads as you need for developing your Helmholtz Cloud Agent. When you run the command you should either see an output with a ResourceCreatedV1 or ErrorV1 message.

Writing a Message Handler

So far, we have successfully sent and received messages from the sample Helmholtz Cloud Agent, but we have not seen how message handling is implemented. The Helmholtz Cloud Agent library provides an application that handles the actual communication with Helmholtz Cloud. You can register handler functions that will be called whenever a message of a certain type is received. This is then the entry point for your development, from where you can then make the necessary calls to provision the resource at your service.

A very simple Helmholtz Cloud Agent would look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
...
from helmholtz_cloud_agent.core.main import HCAApplication
from helmholtz_cloud_agent.messages import ResourceAllocateV1, ResourceCreatedV1
...

app = HCAApplication()

@app.handle(message_type="ResourceAllocateV1")
def resource_allocate_v1(
    correlation_id: str, payload: ResourceAllocateV1
) -> ResourceCreatedV1:
    logger.info(f"Received a message payload of type '{payload.type}'")
    return ResourceCreatedV1(id=str(uuid.uuid4()))

app.run()

We start by importing the HCAApplication that comes with the Helmholtz Cloud Agent library along with the Pydantic structs for the ResourceAllocateV1 and ResourceCreatedV1 messages. Then, we create the actual app and register a function that will handle the messages we receive with the app.handle decorator. A different handler needs to be defined for each message type, and we need to specify it with the message_type parameter. The function itself will always get the correlation ID and the payload. The type will always be the same as the message type.

That’s it. Now you can take the payload and write the code that does the actual resource provisioning.

When the resource is created, you need to send a ResourceCreatedV1 message back to Helmholtz Cloud. The only thing you need to provide is an ID. This ID will be stored by Helmholtz Cloud and when the resource should be deprovisioned, Helmholtz Cloud will send you a deprovisioning request with this ID. Therefor, the ID should uniquely identify the resource at your service.

If for some reason the resource cannot be provisioned, you can send a message back to Helmholtz Cloud that is displayed to the user. To do this, you can define exception classes that can be raised in the message handler. The Helmholtz Cloud Agent application will handle these and send the appropriate message to Helmholtz Cloud.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
...
class ChatTeamAlreadyExistsError(Exception):
    def __init__(
        self: "ChatTeamAlreadyExistsError",
        folder_name: str,
        message: str = "Chat team already exists",
    ) -> None:
        self.folder_name = folder_name
        self.message = message
        super().__init__(self.message)

    def __str__(self: "ChatTeamAlreadyExistsError") -> str:
        return f"{self.message}: {self.folder_name}"
...
@app.handle(message_type="ResourceAllocateV1")
def resource_allocate_v1(
    correlation_id: str, payload: ResourceAllocateV1
) -> ResourceCreatedV1:
...
    raise ChatTeamAlreadyExistsError(spec.desired_name)
...

Deploying your Helmholtz Cloud Agent

When you are ready to deploy your Helmholtz Cloud Agent, the template repository provides a sample Dockerfile that you can use to package your code. It installs a Python environment with all the dependencies from pyproject.toml, takes everything from the src folder and puts it in the /opt/hca folder in the container. Then, you just have to adjust the ENTRYPOINT to invoke your application.

To start the Helmholtz Cloud Agent, you need to set some environment variables:

  • HCA_URL: Your secret URL for connecting your HCA to the cloud. You will get this from Helmholtz Cloud developers.
  • HCA_SERVICE: The ID of your service in the Helmholtz Cloud. You will get this from Helmholtz Cloud developers.