The CloverDX Server offering on AWS Marketplace provides an easy way to create a CloverDX Server instance in the AWS cloud infrastructure. The offering spins-up a recommended cloud architecture that contains a standalone CloverDX Server with good defaults and recommended environment. The server uses PostgreSQL in RDS as its system database, which is automatically created and configured. The server instance uses AWS resources of the user, who is charged for them by Amazon.
The CloverDX Server AWS offering consists of an AMI image of a virtual machine and of a CloudFormation template that provides a simple configuration user interface and creates the required cloud infrastructure as a CloudFormation stack.
The following templates are provided:
CloverDX Server Deployment New VPC - deploys a new instance of CloverDX Server with newly created network infrastructure (VPC, Subnets etc.), see Quickstart. This is the easiest deployment option.
CloverDX Server Deployment Existing VPC - deploys a new instance of CloverDX Server into existing network infrastructure selected by the user, see Deployment Into Existing Infrastructure. This is the option to use if you already have existing infrastructure in AWS and want to integrate CloverDX with it.
CloverDX Server Upgrade - upgrades a CloverDX Server instance to a new version of CloverDX Server, see Upgrade in AWS.
- license for CloverDX Server - this is a BYOL (Bring Your Own License) offering so you need to get your license from us - start here.
- basic familiarity with AWS EC2 infrastructure
- basic familiarity with AWS RDS infrastructure
- basic familiarity with AWS CloudFormation
- account on AWS and permissions to use EC2, RDS and CloudFormation services
- key pair registered in EC2
High-level overview of steps:
- Subscribe to CloverDX Server on AWS Marketplace
- Configure and deploy the CloudFormation stack
- Activate and configure the server instance
Subscribe to CloverDX Data Management Platform - Server BYOL offering on the AWS Marketplace, use the CloverDX Server Deployment New VPC Cloudformation template. Use the Continue to Subscribe button on the offering marketplace page and proceed through the wizard to the Launch action (meanwhile accepting Terms and Conditions, selecting CloverDX version and template etc).
Deploy the CloudFormation stack
First step: Specify template - a CloudFormation template is already selected from the marketplace offering, continue with Next
Second step: Specify stack details - configure the stack:
Figure 11.3. Stack details
- Stack name - unique name of the stack.
- EC2 Instance type - type of EC2 instance where the server will run. A default instance type is pre-selected. You can select one of the supported instance types - larger instances have better performance but at a higher cost.
- EC2 Availability zones - you must select 2 availability zones where the whole stack will be run. The first availability zone will be used for the CloverDX Server EC2 instance, both will be used for the system RDS database. An availability zone is basically a data center. Some availability zones might not support the selected EC2 instance type or the instance type might be unavailable due to capacity reasons - in such case select a different availability zone. Such issue would manifest during startup of the stack in its Events log.
- Key Pair name - select a public key registered in your EC2 infrastructure. This key will be used to administer the EC2 instance via SSH.
- Allow connections from - the CloudFormation template creates a Security Group automatically that allows only connections from specific IP ranges to specific ports on the instance. We recommend that you provide a range of IP addresses from which the instance should be available - typically your offices or data centers. For evaluation purposes you can use your public IP, obtained e.g. from myip.com. We do not recommend making the instance visible to the whole internet.
- Admin user name - specify the user name of CloverDX Server administration user (or keep the default
clover). This user is the first user available in Server Console, for administration of the server itself. This is NOT the operating system user - you must use the above public key to SSH to the instance.
- Admin user password - specify the password for the above admin user.
- Confirm admin user password - re-type the above password to confirm it.
- Database master user name - specify the master user name of the RDS database that will be created as the server's system database. This user name and below password will be used by CloverDX Server to connect to the database.
- Database master user password - specify the password for the above database master user.
- Confirm database master user password - re-type the above password to confirm it.
Configure stack options and review the stack - next 2 pages allow you to set-up additional more advanced stack options, and review the whole stack configuration.
Create stack - click the final Create Stack button to start the stack. Creation of the whole stack takes some time (up to a few minutes). You will see a CloudFormation log of resources being created. The stack is created and ready to use when it gets into the
Figure 11.4. Stack deployment
Success. CloverDX Server is now available in AWS.
You can find its URL in the
Outputs tab of the CloudFormation stack - the
There you can also find its hostname for SSH access.
Figure 11.5. Stack outputs
On the Server’s URL you will see the login page where you can:
Activate the server - the Server is licensed in BYOL (Bring Your Own License) mode, so you need to get your license from us - start here.
To login, use the credentials set in the CloudFormation configuration wizard.
Figure 11.6. CloverDX Server login page
The Server is running with default settings, and is immediately usable. It can be configured further to get it into full production quality.
The CloverDX Server AWS offering consists of an AMI image of a virtual machine and of a CloudFormation template that orchestrates the required cloud resources:
Figure 11.7. Architecture - CloverDX Server in AWS marketplace
Details of the AWS topology:
- The instance runs in the AWS cloud, in the Region selected by user and in its 2 selected Availability Zones. The first AZ is used by the server's EC2 instance, both AZs are used by the RDS database (for potential Multi-AZ deployment, see database details below).
- A new VPC (Virtual Private Cloud) is created to isolate the CloverDX instance from other resources present in the AWS cloud. The VPC has an Internet Gateway for connectivity to the internet (also for the users to connect to the Server Console).
- A new public Subnet is created in the VPC for the server's network resources.
- A single EC2 instance is created from an AMI. The CloverDX Server is running in this instance and uses an RDS instance of PostgreSQL, see below.
- The EC2 instance is available via an Elastic IP - this IP address doesn’t change between restarts of the instance.
- The EC2 instance uses a Security Group to limit access only to specific ports (22, 80, 443) and only from IP addresses from a defined CIDR range.
- 2 new private Subnets are created in the VPC for the RDS database. The database currently uses just one of them, the second one is prepared for multi-AZ deployment of the database.
- An RDS PosgreSQL database is created with recommended settings, and is used as the server's system database.
- The RDS database uses a Security Group to limit access only from the server's EC2 instance.
EC2 instance details (for additional information, see Common cloud architecture):
- Operating system: Amazon Linux 2
- AdoptOpenJDK 11
- Tomcat 9
- 2 disks - OS disk, data disk, both are Provisioned IOPS SSD EBS volumes
RDS database details:
- Database type - PostgreSQL 11.6
- DB instance class - db.m5.large (2 vCPUs, 8GB RAM)
- Storage - 100GB SSD allocated storage, can autoscale to 200GB
- Automated backups enabled - performed every day, backups retained for 7 days
- Available only from the server's EC2 instance, not visible from the Internet
- Server uses SSL to communicate with the database
- Multi-AZ deployment is disabled by default. Multi-AZ deployment runs multiple replicas of the database in different availability zones for high availability, which is why 2 availability zones have to be selected when creating the stack. The architecture and configuration is prepared for Multi-AZ deployment, and its possible to enable it in the RDS configuration later.
For details about CloverDX Server configuration, see Common cloud configuration.
Heap sizes for Server Core and Worker are set automatically based on the instance memory size, see Common cloud memory configuration. It is possible to change the instance type of the VM and the memory sizes will be re-calculated - for this stop the VM instance in EC2 console, change its instance type and start it again.
Users available in the EC2 instance:
ec2-user- standard AWS administrator user of the instance. Use
sudoto run commands that require root privileges. Login via SSH using the public key selected when starting the stack.
root- not recommended to be used, cannot login to it directly.
clover- user that runs the CloverDX Server (i.e. it runs Tomcat). All files that CloverDX uses are owned by the
cloveruser. It is not possible to login as
The EC2 instance uses UTC timezone by default. The instance runs on Amazon Linux, so to change the timezone follow the documentation.
This section describes security related aspects of the CloverDX Marketplace offering.
The CloudFormation template creates a security group that serves as a virtual firewall. The security group allows connections to the following ports:
80- HTTP for Server Console and Server API
443- HTTPS for Server Console and Server API
The connections are allowed only from the IP range specified when configuring the stack during its startup.
The security group settings can be modified - you can find the security group in the Resources section of the CloudFormation stack and click on its configuration.
The CloverDX Server running in the stack has both HTTP and HTTPS enabled by default.
You can find the server’s HTTPS URL in the Outputs section of the CloudFormation stack
ServerHttpsURL key. The HTTPS connector is running on port
The default HTTPS connector is using a self-signed certificate. So it is useful for encryption of communication
between client (e.g. Designer) and the Server, but not for server identity verification. When connecting from
a browser to the Server Console you will first see a warning about the certificate
Your connection is not private, NET::ERR_CERT_AUTHORITY_INVALID), after accepting this
certificate you can work as usual and following accesses will not see this error. When first connecting from
Designer you will need to accept the certificate.
You can use your own production-quality certificate for the HTTPS connector:
- Add your certificate to
/var/clover/conf/serverCertificate.jkskeystore (via the
keytoolcommand), with an alias that is different from
- Modify the
/opt/clover/tomcat/conf/server.xmlTomcat configuration file to use the new certificate alias for its HTTPS connector.
To disable usage of plain HTTP connectivity, modify the security group of
the stack to block all connections to port
Deployment Into Existing Infrastructure
If you’re already operating services and instances in AWS and want to deploy CloverDX Server alongside them, e.g. to integrate several data sources, then the deployment is performed with the CloudFormation template CloverDX Server Deployment Existing VPC. The template does not create a new VPC, Subnets and some other networking infrastructure, but lets you select existing ones. The template creates a new CloverDX Server EC2 instance and a new RDS system database and deploys them into the selected network resources.
Deployment steps and settings in this case are the same as described in Quickstart with these differences:
Subscribe to CloverDX Data Management Platform - Server BYOL offering on the AWS Marketplace - use the CloverDX Server Deployment Existing VPC template
Configure the stack:
Figure 11.8. Deployment to existing infrastructure stack details
VPC - select a VPC where the stack will be deployed.
Subnet - select a Subnet which will be used by the CloverDX Server EC2 instance. The Subnet must be in the VPC selected above.
Database Subnets - select 2 Subnets which will be used by the RDS system database. Typically the subnets are private and not visible from the internet. The subnets must be in the VPC selected above.
Security Group - select a Security Group which will be used by the CloverDX Server EC2 instance. The RDS system database will use its own Security Group that is created by the template and allows connections only from the Server.
To upgrade CloverDX Server to a new version in AWS Marketplace, use the CloverDX Server Upgrade template. The upgrade template creates a new stack with the new version of CloverDX and possibly updated cloud architecture. Configuration and data are re-used from the previous version by using snapshots of data disk and RDS system database. The previous version has only minimal downtime and can run in parallel with the new version.
First see an overview of the upgrade process in Common cloud upgrade. This section assumes familiarity with deploying a new CloverDX Server as described in Quickstart. The upgrade template is also similar to the template for deployment into existing infrastructure (see Deployment Into Existing Infrastructure).
The upgrade process is supported when upgrading from CloverDX version 5.6.1 or newer.
- Create snapshots of the data disk and RDS database of the previous version
- Subscribe to new version of CloverDX Server on AWS Marketplace using the CloverDX Server Upgrade template
- Configure and deploy the CloudFormation stack of the new version, using the above snapshots
- Test the new version
The below steps focus on the new or different actions done during the upgrade process and they assume knowledge of the deployment of a new CloverDX Server stack (see Quickstart).
Pause processing of jobs in the previous version - disable listeners and schedules, this starts downtime of the previous version.
Create a snapshot of the system RDS database of the previous version - you can find the RDS database in the Resources section of your stack. After clicking on it, use the Take Snapshot action on the database.
Figure 11.9. Selecting RDS system database in the stack
Figure 11.10. Creating snapshot of the RDS system database
Create a snapshot of the data disk of the previous version - in the Resources section of your stack, you can find the EC2 instance. After clicking on it, in the EC2 details you can click on the
/dev/sdfvolume to take you to the EBS volume to create the snapshot.
Figure 11.11. Selecting CloverDX Server EC2 instance
Figure 11.12. Selecting data disk EBS volume
Resume processing of jobs in the previous version - this ends the downtime of the previous version.
Subscribe to the new version of CloverDX Data Management Platform - Server BYOL offering on the AWS Marketplace, use the CloverDX Server Upgrade Cloudformation template.
Configure the stack of the new version:
Figure 11.13. Upgrade stack details
VPC - select the VPC where the previous version is running. If you used the CloverDX Server Deployment New VPC template to deploy the previous version, then you can find the VPC by its name “stackname-VPC” (“stackname” is the name of the stack of the previous version).
Subnet - select the Subnet where the previous version EC2 instance with CloverDX Server is running. You can find the Subnet by its name “stackname-Subnet” if you deployed the previous version via the CloverDX Server Deployment New VPC template.
Database Subnets - select the 2 Subnets where the previous version RDS system database is running. You can find the Subnets by their names “stackname-DBPrivateSubnet1” and “stackname-DBPrivateSubnet2” if you deployed the previous version via the CloverDX Server Deployment New VPC template.
Security Group - select the Security Group used by the previous version’s EC2 instance. You can find the Security Group by its name “stackname-SecurityGroup” if you deployed the previous version via the CloverDX Server Deployment New VPC template.
Data disk snapshot - ID of the snapshot of the data disk of the previous version that you created in one of the previous steps. You can find it in the EC2 console in the Snapshots section after selecting the snapshot.
Figure 11.14. Data disk snapshot ID
RDS database snapshot - ARN of the snapshot of the RDS system database of the previous version that you created in one of the previous steps. You can find it in the RDS console in the Snapshots section after selecting the snapshot.
Figure 11.15. RDS system database snapshot ARN
Note that you’re not configuring the credentials of the CloverDX Server admin user and database credentials - they are the same as in the previous version.
Success. New version of CloverDX Server is now available in AWS. You can find its URL in the Outputs tab of the CloudFormation stack. Login credentials for the new version are the same as for the previous version. The new version is deployed in a new version of the cloud architecture, while re-using the data disk and RDS system database of the previous version based on their snapshots. The previous version is left untouched and can be continued to be used.
Activate the new server - you will probably need to provide a license for the new CloverDX Server version. You can do this on the login page.
Test the new version - perform thorough testing of the new version. If you disabled the listeners and schedules on the previous version, then the new version starts up and does not process new data. You need to carefully enable processing that can be enabled, e.g. if it doesn’t consume production data.
After finishing testing of the new version it is possible to switch the Elastic IP address of the previous version to point to the new version instance. This allows you to continue using the same IP address or hostname with the new version, e.g. for Server projects in CloverDX Designer.
Steps to switch Elastic IP address:
Disassociate the new Elastic IP - un-bind the new Elastic IP from the new version instance. Select the Elastic IP of the new version (e.g. from the Resources tab of the stack of the new version) and use the Disassociate Elastic IP address action. Warning: this will cause the new version instance to be available on a different IP address than previously (one will be assigned automatically by AWS).
Release the new Elastic IP - the above disassociated Elastic IP is currently unused and you’re being charged for it. So release it via the Release Elastic IP address action.
Associate the old Elastic IP to the new version instance - to redirect the old Elastic IP address to the new version instance, select it (e.g. from the Resources tab of the stack of the previous version) and use the Associate Elastic IP address. Warning: this will cause the previous version instance to be available on a different IP address than previously (one will be assigned automatically by AWS).
For SSL connectivity to the new version to work correctly, we need to re-generate the self-signed certificate to correspond to the new public hostname. For this just restart the new version instance.