We are setting up our production portal environment. We plan to use VM as portal servers. There is maximum memory limit to those VMs which is 64GB.
Based on the “Portal Mirror Specification” document, current ICGC production setup has some servers with 128GB RAM:
- 1 Varnish server node
- 15 elastic search node
- 32 HDFS nodes
I am wondering if it is possible for us to VMs with 64GB memory to achieve the same performance result.
For elastic search and HDFS, since they are big data app, I assume they are scalable app. To achieve the same result of 128GB server, we can use roughly two 64GB nodes. Is my assumption true from ICGC experience?
For Varnish server, it doesn’t appear there is a scalable cluster solution. There is HA solution which replicate cache between two servers. This is not what we are looking for. There are a couple of potential solutions:
- Just use smaller cache and hope performance is good enough.
- Use physical server. This is more costly and inconvenient.
- Put caching backend on SSD.
From ICGC experience, is 64GB Varnish cache server good enough? Has ICGC tried using SSD as cache?