{"id":923,"date":"2024-06-25T09:11:24","date_gmt":"2024-06-25T07:11:24","guid":{"rendered":"https:\/\/anjo.pt\/keyword-oracle\/?p=923"},"modified":"2024-06-20T12:14:10","modified_gmt":"2024-06-20T10:14:10","slug":"using-ai-to-confirm-a-wrongly-cabled-exadata-switch-or-how-to-fix-verify_roce_cables-py-script-for-python3","status":"publish","type":"post","link":"https:\/\/anjo.pt\/keyword-oracle\/2024\/06\/25\/using-ai-to-confirm-a-wrongly-cabled-exadata-switch-or-how-to-fix-verify_roce_cables-py-script-for-python3\/","title":{"rendered":"Using AI to confirm a wrongly cabled Exadata switch – or how to fix verify_roce_cables.py script for Python3."},"content":{"rendered":"\n

One of the preparation steps when installing an Exadata X10M is to verify that the cabling of the RoCE switches is correctly done. The next step is to upgrade the Cisco switches with the latest firmware. During my intervention for Tradeware at the customer, the first didn’t work as the provided script is not compatible with Python3 and the latter complained about wrong cabling.<\/p>\n\n\n\n

Here I show how studied the wrong cabling of the X10M switches and how I use Claude.ai (ChatGPT and other AI tools probably also work) to quickly fix the Python script provided by Oracle.<\/p>\n\n\n\n\n\n\n\n

When using the verify_roce_cables.py script, I got the following error multiple times:<\/p>\n\n\n

\nibdiagtools# .\/verify_roce_cables.py -n NODES.lst -s SWITCHES.lst\n\nFile "\/usr\/lib64\/python3.6\/multiprocessing\/process.py", line 258, in _bootstrap\n    self.run()\n  File "\/usr\/lib64\/python3.6\/multiprocessing\/process.py", line 93, in run\n    self._target(*self._args, **self._kwargs)\n  File ".\/verify_roce_cables.py", line 12955, in get_vlan_from_host\n    d[host] = [bytes2str(line).strip() for line in ret\n  File ".\/verify_roce_cables.py", line 12956, in <listcomp>\n    if re.search('bound to|ipv4|ipv6', line)]\n  File "\/usr\/lib64\/python3.6\/re.py", line 182, in search\n    return _compile(pattern, flags).search(string)\nTypeError: cannot use a string pattern on a bytes-like object\n<\/pre><\/div>\n\n\n
For the moment I ignored, until I tried to upgrade the switches firmware and got:<\/p>\n\n\n
\npatch_switch_23.1.15.0.0.240605# .\/patchmgr --roceswitches switches.lst --upgrade\n \n2024-06-19 12:35:27 +0200        :Working: Initiate upgrade of 2 RoCE switch(es) to 10.2.4 Expect up to 15 minutes for each switch \n \n2024-06-19 12:35:30 +0200 1 of 2:Running upgrade on switch 10.10.20.21\n \n2024-06-19 12:35:37 +0200:        [INFO     ] Performing Nodes connectivity tests on 10.10.20.21\n2024-06-19 12:35:44 +0200:        [FAIL     ] PORT Eth1\/26 ON SWITCH roceA.client.ch NOT CONNECTED:  Please check connections\n2024-06-19 12:35:44 +0200:        [FAIL     ] PORT Eth1\/25 ON SWITCH 10.10.20.21 NOT CONNECTED:  Please check connections\n2024-06-19 12:35:44 +0200:        [FAIL     ] [FirmwareUpgradeError] Fabric health check failed\n<\/pre><\/div>\n\n\n
Hum, not nice. Based on this I went to the documentation – Cabling Tables for\u00a0Oracle Exadata Database Machine\u00a0X10M<\/a> – and on the Table 13-2 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X10M Servers and Leaf Switches<\/a> I see:<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
The first column is the position of the target compoment.<\/p>\n\n\n\n
The third column the position of the RoCE switch (switch A on U20 switch B on U22).<\/p>\n\n\n\n
The last column says the switch port number.<\/p>\n\n\n\n
So I can see that port 25 and 26 on switch B do not have anything on the other side. Using the first column, the U8 points (usually) to Storage Server 4 and U10 to Storage Server 5. This Exadata has 4 storage servers, so one cable is not right.<\/p>\n\n\n\n
I connect directly to the switch B and there using the command<\/p>\n\n\n
\nswitch# show interface status\n<\/pre><\/div>\n\n\n
The confirmation comes. There is a cable on Eth1\/25 that should go to RU10, where there is no storage server.<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
I felt the “verify_roce_cables.py” would be useful to confirm my assumptions of the cabling. But I do not use Python enough to know what can be the problem. On Oracle Support I found out the problem was related to the Python version:<\/p>\n\n\n\n
Verify_roce_cables.py May Fail with Newer Versions of Python (Doc ID 2971639.1)<\/a><\/p>\n\n\n\n
The provided “solution” doesn’t please me, the Exadata machine is installed in a protected network and bring extra software there is not easy.<\/p>\n\n\n\n
\n\n\n\n
SOLUTION<\/strong><\/p>\n\n\n\n
Use an older version of Python to Run\u00a0verify_roce_cables.py<\/strong>
ibdiagtools#\/usr\/bin\/python2.7 .\/verify_roce_cables.py -n NODES.lst -s SWITCHES.lst<\/code>
The plan is to provide a fix in Exadata image 23.2.0 when it is released<\/p>\n\n\n\n
\n\n\n\n
We are using the latest available 23.1.15 version released 10 days ago provided at Exadata Database Machine and Exadata Storage Server Supported Versions (Doc ID 888828.1)<\/a>.<\/p>\n\n\n\n
But then I though, maybe AI can help me. So I turned to my currently preferred solution claude.ai<\/a> and provided as prompt:<\/p>\n\n\n
\n
\"\"<\/a><\/figure>\n<\/div>\n\n
\nThis script works in Python 2.7 but not in Python 3. The error is the following:\n...\nwhat can the the cause?\n<\/pre><\/div>\n\n\n
Its answer was great, explaining in which line the error was and providing explanations what changed in Python3.<\/p>\n\n\n
\n
\"\"<\/a><\/figure>\n<\/div>\n\n\n
From the three options provided, the first one did not seem right to me. The third would be a catch all but not so nice (why it converts the pattern to bytes like in the first solution before compiling?), so I decide to use the second solution.<\/p>\n\n\n
\n
\"\"<\/a><\/figure>\n<\/div>\n\n\n
I profit from one my preferred thing in Python – the dot notation – and change the line 12956 to:<\/p>\n\n\n
\n        if re.search('bound to|ipv4|ipv6', line.decode('utf-8'))]\n<\/pre><\/div>\n\n\n
Or using sed<\/p>\n\n\n
\ncp verify_roce_cables.py verify_roce_cables.py.orig.$(date +%Y%m%dT%H%M%S)\nsed -i "s\/if re\\.search('bound to|ipv4|ipv6', line)]\/if re\\.search('bound to|ipv4|ipv6', line.decode('utf-8'))]\/" verify_roce_cables.py\n<\/pre><\/div>\n\n\n
Now the script works wonderfully and confirms my findings:<\/p>\n\n\n\n
\"\"<\/p>\n\n\n\n
PS: I’ve checked version included in the Exadata version 24.1.1 (Patch 36651148) and here the Python code was changed, but they use a much more complex solution, with self written “bytes2str” function! Maybe an AI tool would also be useful at Oracle to improve coding. \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the preparation steps when installing an Exadata X10M is to verify that the cabling of the RoCE switches is correctly done. The next step is to upgrade the Cisco switches with the latest firmware. During my intervention for Tradeware at the customer, the first didn’t work as the provided script is not compatible […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[67,69,6],"tags":[],"class_list":{"0":"post-923","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-engineer-systems","7":"category-exadata","8":"category-oracle","9":"czr-hentry"},"_links":{"self":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/923","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/comments?post=923"}],"version-history":[{"count":4,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/923\/revisions"}],"predecessor-version":[{"id":934,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/923\/revisions\/934"}],"wp:attachment":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/media?parent=923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/categories?post=923"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/tags?post=923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}