Background
For every new deployment or storage nodes addition operations, the Elastifile Management Server validates the available quotas, for many required resources such as pd-ssd, cpus, memory, etc.
As part of GCP changes to the Local SSD quota, which described in this article, a new issue introduced when Elastifile queries the response from the GCP API.
Issue
GCP Compute API changed the way they return the response, and the Elastifile Management Server unable to parse this new response.
Hence, whenever an add node operation is executed, the following error might be shown and the user could not add the new resources:
In the /elastifile/log/ecp/cloud_provider.log , you should find the following error:
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/thrift/server/TServer.py", line 167, in serveClient
self.processor.process(iprot, oprot)
File "/elastifile/ecp/app/api/ecp_api/ECPService.py", line 1226, in process
self._processMap[name](self, seqid, iprot, oprot)
File "/elastifile/ecp/app/api/ecp_api/ECPService.py", line 1423, in process_quota_get
result.write(oprot)
File "/elastifile/ecp/app/api/ecp_api/ECPService.py", line 4811, in write
self.success.write(oprot)
File "/elastifile/ecp/app/api/ecp_api/ttypes.py", line 3361, in write
iter92.write(oprot)
File "/elastifile/ecp/app/api/ecp_api/ttypes.py", line 1888, in write
oprot.writeI64(self.limit)
File "/usr/lib64/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py", line 114, in writeI64
buff = pack("!q", i64)
error: integer out of range for 'q' format code
Solution
New Deployment
The following steps should be taken:
1. SSH to the EMS
root@patch ~# . elfs_admin
[root@patch ~(elfs_admin)]#
2. Edit the file named /elastifile/ecp/app/lib/ecp_gce.py and comment out the following lines:
# quota = dict(metric='local-ssd',
# usage=int(item.get('usage')),
# limit=int(item.get('limit')))
# quotas.append(quota)
3. Restart the ECP service: systemctl restart ecp
4. Complete the new deployment
5. Apply the patch which described in the next section: Adding Nodes on Running System
Adding Nodes on Running System
Elastifile offers the following patch that can be installed by the user in order to solve the issue.
For that, follow the steps below:
1. SSH to the EMS
root@patch ~# . elfs_admin
[root@patch ~(elfs_admin)]#
2. Download the required patch
3. Upload the patch to EMS software repo
4. Install the patch
Validation
In order to validate that the patch installed successfully and the issue is resolved, run the following command and expect a valid output:
{
"meta": {
"msg": []
},
"quotas": [
{
"limit": 600,
"metric": "cores",
"usage": 24
},
{
"limit": 102400,
"metric": "external-hdd",
"usage": 300
},
{
"limit": 575,
"metric": "ip-addresses",
"usage": 3
},
{
"limit": 40960,
"metric": "external-ssd",
"usage": 2820
},
{
"limit": 6000,
"metric": "instances",
"usage": 6
},
{
"limit": 4094,
"metric": "internal-ip-addresses",
"usage": 8
},
{
"limit": 500,
"metric": "firewalls",
"usage": 61
},
{
"limit": 300,
"metric": "routes",
"usage": 35
}
]
}