SyncOps: An Orchestration Engine of SyncHPC

Do you know how SyncHPC works behind the scenes? For those immersed in the world of HPC / AI or VDI, understanding the inner workings of SyncHPC can unlock unlimited possibilities. SyncHPC platform is designed to deploy big compute workloads on cloud or on-premises. In this blog, we’ll explain the orchestration engine of SyncHPC, exploring its role in overall architecture and key features.

SyncOps is an integral part of SyncHPC. It is used to orchestrate the deployment of resources. SyncOps does 3 major tasks: Deploy, Destroy and Update. Users can provide an input with series of operations, their dependencies and inputs variables. It also takes the ‘credentials’ required to access cloud or on-premises resources.

One of the interesting parts of SyncOps is that it can trigger any type of orchestration framework. For example, it supports Terraform, Ansible, Simple Shell Scripts, Azure Templates, or even any other deployment automation framework. To add a new automation framework, SyncOps expects to write a python based simple plugin. Users can write their own provider automation and create a plugin in SyncOps. Interestingly, the input for SyncOps is in defined template format. Hence, even if you want to trigger different types of automation, the input for SyncOps will be of same type. Hence, there is a single engine (SyncOps), which can trigger any type of deployment automation. Blog post image clarifies it.

Please note that SyncHPC already has a large number of plugins. E.g. automation deployments of HPC/AI and VDI on cloud and on-premise is already taken care of. But, users can still create their own plugins. For example, if user wants to add a new cloud provider for AI workloads, then adding is very easy.

Suppose there are 3 automation users and they want to add their own automation in SyncOps.

• First user creates an automation in Terraform to deploy AI framework in Google Cloud.
• Second user creates an automation in Terraform to deploy HPC cluster in any other cloud.
• Third user creates an automation with Azure Templates to deploy something on Azure.

Now, there can be 3 new plugins. Although first 2 users are using Terraform, they can still create their own 2 different plugins. This can simplify their solution to deploy/update with specific inputs.

Now, it should be nearly clear how SyncHPC utilises the SyncOps component. SyncOps already has plugins created for supported workloads. User selects a specific deployment options on SyncHPC Web Portal. Then, SyncHPC converts the user provided deployment inputs from Web UI to SyncOps readable inputs template. Then, based on selected SyncOps plugin the deployment starts. SyncOps provides the output of each operation and logs of deployment. SyncHPC captures this output to show the progress of deployment automation.

While detailed user guide documentation will help users to use SyncOps, this blog post helps to get an overview of SyncOps, its role and important features.