{"id":966,"date":"2025-01-13T16:41:40","date_gmt":"2025-01-13T15:41:40","guid":{"rendered":"https:\/\/anjo.pt\/keyword-oracle\/?p=966"},"modified":"2025-01-13T16:41:41","modified_gmt":"2025-01-13T15:41:41","slug":"slow-starting-impdp-from-nfs-share","status":"publish","type":"post","link":"https:\/\/anjo.pt\/keyword-oracle\/2025\/01\/13\/slow-starting-impdp-from-nfs-share\/","title":{"rendered":"Slow starting impdp from NFS Share"},"content":{"rendered":"\n<p>Unfortunately I lost the logs for this issue, but I try to document for information.<\/p>\n\n\n\n<p>My customer has ExaCC with various 2-node clusters.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Export ACFS mount point as NFS from cluster1<\/li>\n\n\n\n<li>Mount NFS mount point on cluster2, cluster3 and cluster4<\/li>\n<\/ul>\n\n\n\n<p>He did an export from cluster1 to the ACFS mount point.<\/p>\n\n\n\n<p>All was working fine until mid December, when impdp reading a dumpfile from the NFS mount point seems hanging when was called from cluster3 and cluster4. From cluster2 it was still fine.<\/p>\n\n\n\n<p>Few days later, the impdp was slow everywhere, except locally on cluster1.<\/p>\n\n\n\n<p>The behavior was very bizarre:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>impdp starting showing timestamp<\/li>\n\n\n\n<li>exactly 5 minutes later first output comes &#8220;W-1 Startup took 1 second&#8221;<\/li>\n\n\n\n<li>exactly 5 minutes after comes second line &#8220;W-1 Master table &#8230; successfully loaded\/unloaded&#8221;<\/li>\n\n\n\n<li>and 5 minutes later runs the rest, quickly. <\/li>\n<\/ul>\n\n\n\n<p>The NFS mount point seemed ok, &#8216;dd&#8217; command tests did not show any slowness.<\/p>\n\n\n\n<p>I started to investigating by <a href=\"https:\/\/dohdatabase.com\/2023\/11\/27\/how-to-trace-oracle-data-pump\/\">enabling the DataPump tracing, as explained by Daniel Hansen on his Databases are Fun blog<\/a>:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\nalter system set events &#039;sql_trace {process: pname = dw | process: pname = dm} level=8&#039;;\n<\/pre><\/div>\n\n\n<p>The trace files generated on Diagnostics directory did not help much &#8211; they are mostly for performance problems.<\/p>\n\n\n\n<p>Then I did start a &#8220;strace&#8221; on the PID of the impdp<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\nstrace -p &amp;lt;pid&gt; -o \/tmp\/strace.out\n<\/pre><\/div>\n\n\n<p>There I could see some &#8220;ECONNREFUSED&#8221; to one of the IPs of the Cluster1. But few lines above, there was the same connection without error. <\/p>\n\n\n\n<p>Quite strange. Finally with the help of one system administrator, we found out that the nfs-server was not running on one of the cluster1 nodes. And the NFS mount point was using a hostname which dynamically would go either to one or another node of the cluster1. After making sure nfs-server was running on both nodes from cluster1, the problem was solved and impdp was fast to start again.<\/p>\n\n\n\n<p>Learnings:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use the clusterware to manage exportfs &#8211; srvctl add exportfs<\/li>\n\n\n\n<li>Make use of VIPs which move from one node to another instead of round-robin DNS entries.<\/li>\n\n\n\n<li><\/li>\n<\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Unfortunately I lost the logs for this issue, but I try to document for information. My customer has ExaCC with various 2-node clusters. He did an export from cluster1 to the ACFS mount point. All was working fine until mid December, when impdp reading a dumpfile from the NFS mount point seems hanging when was [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,23,67,82],"tags":[],"class_list":{"0":"post-966","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-datapump","7":"category-dns","8":"category-engineer-systems","9":"category-exacc","10":"czr-hentry"},"_links":{"self":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/comments?post=966"}],"version-history":[{"count":1,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/966\/revisions"}],"predecessor-version":[{"id":967,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/posts\/966\/revisions\/967"}],"wp:attachment":[{"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/media?parent=966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/categories?post=966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/anjo.pt\/keyword-oracle\/wp-json\/wp\/v2\/tags?post=966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}