<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>It all begins with data – It all begins with data</title>
    <link>https://www.beginswithdata.com/</link>
    <description>Recent content on It all begins with data</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en</language>
    <lastBuildDate>Tue, 22 Sep 2026 00:00:00 +0100</lastBuildDate>
    
	  <atom:link href="https://www.beginswithdata.com/index.xml" rel="self" type="application/rss+xml" />
    
    
      
      
    
    
    <item>
      <title>Private GCS via Internal Load Balancers: No Client Auth Required</title>
      <link>https://www.beginswithdata.com/2026/09/22/private-gcs-via-ilb/</link>
      <pubDate>Tue, 22 Sep 2026 00:00:00 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2026/09/22/private-gcs-via-ilb/</guid>
      <description>
        
        
        &lt;p&gt;For web serving use cases it&amp;rsquo;s common to put a load balancer in front of Cloud Storage providing a custom domain name with TLS, path flexibility with a URL map, and a single place to apply policy.&lt;/p&gt;
&lt;p&gt;External Application Load Balancers (ALB) can use backend buckets that are &lt;a href=&#34;https://docs.cloud.google.com/load-balancing/docs/https/setup-global-ext-https-buckets#enable_public_bucket_access&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;shared publicly&lt;/a&gt;, or are &lt;a href=&#34;https://docs.cloud.google.com/load-balancing/docs/https/setup-global-ext-https-buckets#enable_private_bucket_access&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;kept private&lt;/a&gt; by granting a Google managed LB service account access. Many organizations use the private option to prevent direct API access to the bucket by unauthenticated users, as well as to avoid org policy exceptions for &lt;a href=&#34;https://docs.cloud.google.com/storage/docs/public-access-prevention&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;public access prevention&lt;/a&gt; and security findings that come with &lt;code&gt;allUsers&lt;/code&gt; access.&lt;/p&gt;
&lt;p&gt;Visually, the private option looks like this, where the External ALB adds authentication:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2026-09-22/1.png&#34; alt=&#34;Blog solution&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Great, that works for External ALBs, but what about Internal ALBs?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Unfortunately as of today there is no private option per the &lt;a href=&#34;https://docs.cloud.google.com/load-balancing/docs/l7-internal/setup-regional-internal-buckets#limitations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Limitations&lt;/a&gt;: &lt;code&gt;Private bucket access isn&#39;t supported, so the backend bucket must be publicly accessible over the internet.&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;It sounds crazy that the bucket must be publicly available on the Internet in order to share it publicly on our internal network, but this appears to be the situation today. We can use &lt;a href=&#34;https://docs.cloud.google.com/storage/docs/ip-filtering-overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bucket IP filtering&lt;/a&gt; to block Internet access to the bucket, but from an IAM perspective it will still be open. Not good enough. Time to get cracking and see if we can build a solution that keeps the bucket private ourselves!&lt;/p&gt;
&lt;p&gt;This is what we get out of the box with no auth headers added anywhere and a &lt;code&gt;401&lt;/code&gt; response from the private bucket:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2026-09-22/2.png&#34; alt=&#34;Out of box configuration&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;To fix this up we&amp;rsquo;ll need to add auth headers ourselves. Using a &lt;strong&gt;traffic extension&lt;/strong&gt; on the forwarding rule, we&amp;rsquo;ll call a small Go &lt;code&gt;ext_proc&lt;/code&gt; service running on Cloud Run for every request. The service holds a Cloud Storage scoped OAuth token in memory and returns a header mutation that sets &lt;code&gt;authorization: Bearer &amp;lt;token&amp;gt;&lt;/code&gt;. The load balancer applies the header, GCS authenticates the request, and the client gets the data being none the wiser:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2026-09-22/3.png&#34; alt=&#34;Solution configuration using &#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;No signed URLs, no service account keys, no proxy VMs. Clients just fetch objects, and the bucket stays private.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;turning-theory-into-practice&#34;&gt;Turning theory into practice&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;turning-theory-into-practice&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#turning-theory-into-practice&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; fill=&#34;currentColor&#34; viewBox=&#34;3 3 18 18&#34;&gt;
  &lt;path d=&#34;M12 3C7.0275 3 3 7.12937 3 12.2276C3 16.3109 5.57625 19.7597 9.15374 20.9824C9.60374 21.0631 9.77249 20.7863 9.77249 20.5441C9.77249 20.3249 9.76125 19.5982 9.76125 18.8254C7.5 19.2522 6.915 18.2602 6.735 17.7412C6.63375 17.4759 6.19499 16.6569 5.8125 16.4378C5.4975 16.2647 5.0475 15.838 5.80124 15.8264C6.51 15.8149 7.01625 16.4954 7.18499 16.7723C7.99499 18.1679 9.28875 17.7758 9.80625 17.5335C9.885 16.9337 10.1212 16.53 10.38 16.2993C8.3775 16.0687 6.285 15.2728 6.285 11.7432C6.285 10.7397 6.63375 9.9092 7.20749 9.26326C7.1175 9.03257 6.8025 8.08674 7.2975 6.81794C7.2975 6.81794 8.05125 6.57571 9.77249 7.76377C10.4925 7.55615 11.2575 7.45234 12.0225 7.45234C12.7875 7.45234 13.5525 7.55615 14.2725 7.76377C15.9937 6.56418 16.7475 6.81794 16.7475 6.81794C17.2424 8.08674 16.9275 9.03257 16.8375 9.26326C17.4113 9.9092 17.76 10.7281 17.76 11.7432C17.76 15.2843 15.6563 16.0687 13.6537 16.2993C13.98 16.5877 14.2613 17.1414 14.2613 18.0065C14.2613 19.2407 14.25 20.2326 14.25 20.5441C14.25 20.7863 14.4188 21.0746 14.8688 20.9824C16.6554 20.364 18.2079 19.1866 19.3078 17.6162C20.4077 16.0457 20.9995 14.1611 21 12.2276C21 7.12937 16.9725 3 12 3Z&#34;&gt;&lt;/path&gt;
&lt;/svg&gt;
&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;The full walkthrough, the Go source code, and &lt;code&gt;gcloud&lt;/code&gt; commands to set it up are in this GitHub repository:&lt;br&gt;
&lt;strong&gt;&lt;a href=&#34;https://github.com/dutchiechris/storage/tree/main/gcs/gcs-private-bucket-callout&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github.com/dutchiechris/storage/tree/main/gcs/gcs-private-bucket-callout&lt;/a&gt;&lt;/strong&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

The repo has steps to do it all in under 30 minutes:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Private bucket and identity:&lt;/strong&gt; Bucket with public access prevention enforced and the &lt;code&gt;objectViewer&lt;/code&gt; role granted to a service account.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Callout service:&lt;/strong&gt; Under 400 lines of Go in a 5.4 MB distroless image, deployed to potentially dozens of tiny 1 vCPU / 128 MiB Cloud Run instances.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Load balancer:&lt;/strong&gt; A serverless NEG and backend service for the callout, a backend bucket for GCS, and a regional internal forwarding rule in front of both.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Traffic extension:&lt;/strong&gt; Attach the callout extension to the forwarding rule and configure it to process request headers only.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Optional HTTPS:&lt;/strong&gt; Add a private CA, a regional managed certificate, and a private DNS zone if you want TLS and a friendly hostname.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;For the how and why details, check the repo link above.&lt;/strong&gt; The rest of this post are my &amp;ldquo;lessons learned&amp;rdquo;.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;lessons-learned&#34;&gt;Lessons learned&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;lessons-learned&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#lessons-learned&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;&lt;strong&gt;1 - Don&amp;rsquo;t call the metadata server on the request path.&lt;/strong&gt; The Cloud Run metadata server caches tokens well enough to call it per request. Or so I thought. But checking response time metrics after a 20k request test run I found response times for this cached token were 2.5 ms avg, but up to nearly 189 ms at worst. Caching the token in service RAM reduces this to under a microsecond. Considering the other steps in the callout flow take about 6 ms avg, adding latency for the token fetch itself, especially at the tail, would be impactful. Better to refresh the token in the background and serve from memory.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2 - The metadata server won&amp;rsquo;t mint a fresh token until the old one is nearly expired.&lt;/strong&gt; The first request for a token is valid for 30 minutes. Request another five minutes later and you get that same token with 25 minutes remaining. So at what point does the metadata server mint a new one? Through my testing, at &lt;strong&gt;4 minutes remaining&lt;/strong&gt;. I added refresh logic to save it to service RAM at 3 minutes remaining to avoid slowing any client request.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3 - Go sees 2 CPUs, but it turned out not to matter.&lt;/strong&gt; &lt;code&gt;runtime.NumCPU()&lt;/code&gt; returns 2 on 1 vCPU and sub-vCPU instances because the container is scheduled across two logical hyperthreads. However, on a 1 vCPU allocation, if your Go runtime runs two OS threads concurrently at full load, it will be throttled introducing latency jitter, degrading throughput, and potentially triggering &lt;a href=&#34;https://docs.cloud.google.com/run/docs/about-instance-autoscaling#adaptive-concurrency&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Adaptive Concurrency Tuning (ACT)&lt;/a&gt; that reduces request concurrency. Suspecting this was hurting performance, I wrote code to inspect cgroup quotas and set Go&amp;rsquo;s &lt;code&gt;GOMAXPROCS&lt;/code&gt; accordingly.&lt;/p&gt;
&lt;p&gt;While testing with Cloud Run Gen1 (gVisor-based virtualization), I found that &lt;code&gt;cpu.cfs_quota_us&lt;/code&gt; did not reflect limits accurately when &lt;a href=&#34;https://docs.cloud.google.com/run/docs/configuring/services/cpu#startup-boost&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CPU boost&lt;/a&gt; was enabled; it continued reporting the boosted quota even after the startup period expired. Looking for other options, I found &lt;code&gt;cpu.shares&lt;/code&gt; reliably showed &lt;code&gt;1024&lt;/code&gt; shares per configured vCPU regardless of boost (e.g., &lt;code&gt;512&lt;/code&gt; for 0.5 vCPU, &lt;code&gt;2048&lt;/code&gt; for 2 vCPU).&lt;/p&gt;
&lt;p&gt;Under Cloud Run Gen2 (microVM-based execution), the behavior flipped: &lt;code&gt;cpu.cfs_quota_us&lt;/code&gt; accurately tracked the active CPU quota, decreasing after the boost period ended. But now &lt;code&gt;cpu.shares&lt;/code&gt; appeared to be fixed at &lt;code&gt;1024&lt;/code&gt; regardless of instance size or boost state.&lt;/p&gt;
&lt;p&gt;This was starting to get quite complicated, so I decided to measure the impact of setting &lt;code&gt;GOMAXPROCS&lt;/code&gt; to 1 or 2 in the first place. For my workload, I couldn&amp;rsquo;t see any difference: same scaled instance count, comparable CPU utilization, and equivalent response times. So I deleted the code. Tuning likely pays off when you have more CPUs or are purposefully saturating CPU, but for this workload where 1 vCPU runs at around 30% utilization, it doesn&amp;rsquo;t really matter.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;4 - The solution scales up smoothly, and the additional callout latency is minor.&lt;/strong&gt; With a 32 KiB file and 100 concurrent clients fetching it from a GCE VM in the same region I observed around 3,800 req/sec @ 25 ms avg response time with 6 Cloud Run instances active. Quadrupling the clients to 400 increased serving to 12,750 req/sec @ 28 ms avg response time with 14 instances serving. At both load points the callout latency contributed under 6 ms to the overall latency and I didn&amp;rsquo;t see a single HTTP error. These results are once things were warmed up, but even from a cold start (e.g. &lt;code&gt;--min-instances=0&lt;/code&gt;) the very first request had total latency of 342 ms, and the immediate requests thereafter at around 70 ms:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2026-09-22/logs.png&#34; alt=&#34;Cloud Logging request and scaling logs&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;5 - Stop guessing why Cloud Run scaled.&lt;/strong&gt; Coming from an infra background I was interested in testing different Cloud Run shapes to find the best price/perf. Initially, my approach was to test with &lt;code&gt;--min-instances=1&lt;/code&gt; to see how much throughput, and at what latency, I could get from a single instance. Next, I&amp;rsquo;d extrapolate the needed instance count for my workload. &lt;strong&gt;Don&amp;rsquo;t do this.&lt;/strong&gt; Cloud Run&amp;rsquo;s autoscaler is more advanced and uses multiple signals to scale and distribute work. Instead of the extrapolation, start your benchmark and then check the Cloud Monitoring &lt;strong&gt;recommended instances&lt;/strong&gt; metric label &lt;code&gt;scaling_driver&lt;/code&gt;. It will show you how many instances are needed on a per scaling driver basis. During tests with 100 and 400 clients &lt;code&gt;system&lt;/code&gt; was always the greatest value, and two scaling elements I&amp;rsquo;d been investigating were off the table: neither &lt;code&gt;--concurrency&lt;/code&gt; nor &lt;code&gt;act&lt;/code&gt; was anywhere near producing a scaling decision:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2026-09-22/graph.png&#34; alt=&#34;Cloud Monitoring recommended instances, broken down by scaling reason&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;6 - Upsizing resources bought nothing.&lt;/strong&gt; I doubled vCPU to use a 2 vCPU / 128 MiB instance to check the impact on latency, per-instance throughput, and instance count scaling. First, the &lt;code&gt;System&lt;/code&gt; &lt;code&gt;scaling_factor&lt;/code&gt; was unchanged and still leading, so it woudln&amp;rsquo;t run any fewer instances. Next, I analyzed service latency and throughput, and they were comparable too. Usually bigger is better, but in this case it made no difference and would nearly double the cost.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;7 - As request rates get higher, the bucket will be your temporary constraint.&lt;/strong&gt; Cloud Storage &lt;a href=&#34;https://docs.cloud.google.com/storage/docs/request-rate&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ramps a bucket up gradually&lt;/a&gt;, starting at around 5,000 object reads per second. In my test I naturally scaled past 5,000 and hit some errors, &lt;code&gt;429&lt;/code&gt;s and &lt;code&gt;5xx&lt;/code&gt;s mostly, while the bucket was ramping. If you see higher latency or errors be sure to evaluate the full chain: ALB, Callout, and Storage bucket. And especially when testing request bursts, or higher request levels, make sure your bucket is warm.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;what-about-security&#34;&gt;What about security?&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;what-about-security&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#what-about-security&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;The Envoy service extension does not provide any authentication when streaming requests to the callout. As a consequence, the Cloud Run service has to be deployed with &lt;code&gt;--allow-unauthenticated&lt;/code&gt;. To reduce access make it internal using &lt;code&gt;--ingress=internal&lt;/code&gt; and remove endpoints using &lt;code&gt;--no-default-url&lt;/code&gt;. Even so, any client inside the VPC that can reach the callout can get a token back. To limit the blast radius, make the token itself less useful:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Least privilege IAM:&lt;/strong&gt; Grant the service account read access to the one bucket and nothing else.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Downscoped token:&lt;/strong&gt; Scope the token to &lt;code&gt;devstorage.read_only&lt;/code&gt; to ensure even if permissions are expanded the token can&amp;rsquo;t be used with them.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 data-hextra-search-id=&#34;conclusion&#34;&gt;Conclusion&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;conclusion&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#conclusion&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;This solution leverages specific features of storage, networking, and serverless compute to meet a business requirement. While the code that adds an auth token will hopefully be replaced by a built-in feature soon (like it is for External ALBs), the callout design pattern itself is adaptable for other use cases where headers need to be manipulated, or request details logged.&lt;/p&gt;
&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; fill=&#34;currentColor&#34; viewBox=&#34;3 3 18 18&#34;&gt;
  &lt;path d=&#34;M12 3C7.0275 3 3 7.12937 3 12.2276C3 16.3109 5.57625 19.7597 9.15374 20.9824C9.60374 21.0631 9.77249 20.7863 9.77249 20.5441C9.77249 20.3249 9.76125 19.5982 9.76125 18.8254C7.5 19.2522 6.915 18.2602 6.735 17.7412C6.63375 17.4759 6.19499 16.6569 5.8125 16.4378C5.4975 16.2647 5.0475 15.838 5.80124 15.8264C6.51 15.8149 7.01625 16.4954 7.18499 16.7723C7.99499 18.1679 9.28875 17.7758 9.80625 17.5335C9.885 16.9337 10.1212 16.53 10.38 16.2993C8.3775 16.0687 6.285 15.2728 6.285 11.7432C6.285 10.7397 6.63375 9.9092 7.20749 9.26326C7.1175 9.03257 6.8025 8.08674 7.2975 6.81794C7.2975 6.81794 8.05125 6.57571 9.77249 7.76377C10.4925 7.55615 11.2575 7.45234 12.0225 7.45234C12.7875 7.45234 13.5525 7.55615 14.2725 7.76377C15.9937 6.56418 16.7475 6.81794 16.7475 6.81794C17.2424 8.08674 16.9275 9.03257 16.8375 9.26326C17.4113 9.9092 17.76 10.7281 17.76 11.7432C17.76 15.2843 15.6563 16.0687 13.6537 16.2993C13.98 16.5877 14.2613 17.1414 14.2613 18.0065C14.2613 19.2407 14.25 20.2326 14.25 20.5441C14.25 20.7863 14.4188 21.0746 14.8688 20.9824C16.6554 20.364 18.2079 19.1866 19.3078 17.6162C20.4077 16.0457 20.9995 14.1611 21 12.2276C21 7.12937 16.9725 3 12 3Z&#34;&gt;&lt;/path&gt;
&lt;/svg&gt;
&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;The full walkthrough, the Go source code, and &lt;code&gt;gcloud&lt;/code&gt; commands to set it up are in this GitHub repository:&lt;br&gt;
&lt;strong&gt;&lt;a href=&#34;https://github.com/dutchiechris/storage/tree/main/gcs/gcs-private-bucket-callout&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github.com/dutchiechris/storage/tree/main/gcs/gcs-private-bucket-callout&lt;/a&gt;&lt;/strong&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;Use a disposable project, run through it, and see how it fits your environment! As always, comments welcome below.&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Building Serverless Rclone Data Transfers to Google Cloud Storage</title>
      <link>https://www.beginswithdata.com/2026/07/28/rclone-gcs-serverless/</link>
      <pubDate>Tue, 28 Jul 2026 00:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2026/07/28/rclone-gcs-serverless/</guid>
      <description>
        
        
        &lt;p&gt;Google Cloud Storage (GCS) is the common storage layer for your apps on Google Cloud. But, data gravity is real, the world of storage is large and diverse, and at some point you will need to integrate with SFTP, HTTP, S3, WebDAV, Azure Blob, and more.  Google&amp;rsquo;s &lt;a href=&#34;https://docs.cloud.google.com/storage-transfer/docs/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Storage Transfer Service (STS)&lt;/a&gt; supports the most common services, but when it doesn&amp;rsquo;t, the open-source project &lt;a href=&#34;https://rclone.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rclone&lt;/a&gt; can often fill the gap.&lt;/p&gt;
&lt;p&gt;In this post you will learn how to use the &lt;strong&gt;rclone&lt;/strong&gt; open-source data transfer engine to build an automated, zero-maintenance transfer service that transfers data between external services and Google Cloud Storage. Recipes are provided for &lt;strong&gt;SFTP server to GCS&lt;/strong&gt;. Swapping in other backends shouldn&amp;rsquo;t be a problem once you understand the solution building blocks, and in fact at the end is a bonus recipe for &lt;code&gt;HTTP&lt;/code&gt;.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;data-transfer-solutions-with-google-cloud-storage&#34;&gt;Data transfer solutions with Google Cloud Storage&lt;/h1&gt;&lt;p&gt;There are endless ways to transfer data into and out of GCS, but my two favorites are the &lt;strong&gt;Storage Transfer Service&lt;/strong&gt; and &lt;strong&gt;rclone&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Storage Transfer Service&lt;/strong&gt;: Use this fully managed solution when using one of these sources: Amazon S3, generic S3-compatible object stores, Azure Blob Storage, on-premises file systems (via dedicated STS agents), and GCS bucket synchronization. STS is designed for transfers at scale and parallelizes the discovery and transfer phases, has validation checks, is managed with IAM permissions, and has no service costs for most transfers.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;rclone&lt;/strong&gt;: Use rclone when transferring with any storage service not supported by STS such as (S)FTP servers, WebDAV endpoints, SMB/CIFS shares, and file sharing services like OneDrive, Box, Dropbox, and Google Drive. Rclone also supports GCS as the source &lt;em&gt;or&lt;/em&gt; destination, so if you need to transfer data out of GCS to AWS S3 or Azure Blob it can also do the job.&lt;/li&gt;
&lt;/ul&gt;
&lt;blockquote&gt;
  &lt;p&gt;TLDR; If STS handles your use case, use it. When it doesn&amp;rsquo;t, rclone to the rescue!&lt;/p&gt;

&lt;/blockquote&gt;
&lt;h1 data-hextra-search-id=&#34;serverless-rclone-data-transfers-cloud-run-jobs-vs-batch&#34;&gt;&lt;strong&gt;Serverless rclone data transfers: Cloud Run jobs vs Batch&lt;/strong&gt;&lt;/h1&gt;&lt;p&gt;While you can provision a VM, install rclone on it, and configure a cronjob for recurring transfers, there are much cleaner, cheaper, and more secure ways to do it using ephemeral compute and a job-oriented approach. I present two solutions, one using &lt;strong&gt;Cloud Run jobs&lt;/strong&gt; for ultimate simplicity, and another using &lt;strong&gt;Batch&lt;/strong&gt; when more control over compute resources is needed. While the execution engines are different, both designs use the official &lt;strong&gt;rclone&lt;/strong&gt; container image, &lt;strong&gt;Secret Manager&lt;/strong&gt; for protecting SSH keys and &lt;strong&gt;Cloud Scheduler&lt;/strong&gt; for triggering transfer updates.&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Cloud Run jobs&lt;/strong&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Batch&lt;/strong&gt;&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;When to choose&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Ideal for smaller transfers where transfer speeds up to ~30 MiB/s are fast enough.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Ideal for larger transfers where transfer speeds of 500 MiB/s or more are needed.&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Why it shines&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Extremely fast startup times, fully serverless, simple design and metrics and logging across executions.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Flexibility to define VM details fitting a broader set of requirements.&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Why not&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Throughput is too low.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Slower start-up times, solution requires more services, adding complexity. Metrics and monitoring views are per execution, to view across executions requires building own dashboards and log filter views.&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;h3 data-hextra-search-id=&#34;cloud-run-jobs&#34;&gt;Cloud Run jobs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;cloud-run-jobs&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#cloud-run-jobs&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/cloud-run.png&#34; title=&#34;Cloud Run jobs diagram&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Run jobs diagram&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;Cloud Run&lt;/strong&gt; approach is quite straightforward:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Cloud Scheduler&lt;/strong&gt; triggers &lt;strong&gt;Cloud Run job&lt;/strong&gt; on a cron based schedule.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Cloud Run job&lt;/strong&gt; pulls &lt;code&gt;rclone&lt;/code&gt; container from Docker hub.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;rclone.conf&lt;/code&gt; and SSH key are pulled from &lt;strong&gt;Secret Manager&lt;/strong&gt; and mounted into container.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;rclone&lt;/code&gt; process runs, reading and writing data. Logs and metrics (CPU, RAM, network bytes, etc) are saved to the Cloud Run job instance in Cloud Monitoring and Logging.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Wondering about that low 30 MiB/s limit? It comes from Cloud Run&amp;rsquo;s shared network &lt;a href=&#34;https://docs.cloud.google.com/run/quotas#bandwidth_limits&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bandwidth limit&lt;/a&gt; of 600 Mbps (ingress+egress). While you can use Direct VPC to achieve greater throughput, the NAT processing charge for inbound data adds up. Better to use a different service to avoid it, or shard your transfers across more jobs to stay within it.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;batch&#34;&gt;Batch&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;batch&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#batch&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/batch.png&#34; title=&#34;Batch diagram&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Batch diagram&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;Using &lt;strong&gt;Batch&lt;/strong&gt; has these steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Cloud Scheduler&lt;/strong&gt; triggers a &lt;strong&gt;Workflow&lt;/strong&gt; using a cron based schedule.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Workflow&lt;/strong&gt; creates a &lt;strong&gt;Batch&lt;/strong&gt; job that includes the &lt;code&gt;rclone&lt;/code&gt; and VM worker configuration.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Batch&lt;/strong&gt; pulls &lt;code&gt;rclone&lt;/code&gt; container from Docker hub.&lt;/li&gt;
&lt;li&gt;The SSH key is pulled from &lt;strong&gt;Secret Manager&lt;/strong&gt; and saved to the VM.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;rclone&lt;/code&gt; process runs on the exact VM shape requested, reading and writing data. Logs and metrics (cpu, ram, network bytes, etc) are saved to the VM instance in Cloud Monitoring and Logging.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;With &lt;strong&gt;Workflow&lt;/strong&gt; and &lt;strong&gt;Batch&lt;/strong&gt; you have more control over the transfer resources and can build more sophisticated workflows.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;recipes-cloud-run-jobs--batch&#34;&gt;Recipes: Cloud Run Jobs &amp;amp; Batch&lt;/h1&gt;&lt;p&gt;Let&amp;rsquo;s make this real with recipes for each of these options!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;recipe-1-cloud-run-jobs&#34;&gt;Recipe 1: Cloud Run jobs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;recipe-1-cloud-run-jobs&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#recipe-1-cloud-run-jobs&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Goal: Use Cloud Run jobs to copy files from a remote SFTP server to a GCS bucket every two hours using a lightweight, zero-maintenance Cloud Run job. These steps assume you are authenticated from a command prompt with &lt;code&gt;gcloud&lt;/code&gt; and a default project has been selected. Using &lt;code&gt;Cloud Shell&lt;/code&gt; is an option.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;1-set-up-environment-variables--enable-apis&#34;&gt;(1) Set Up Environment Variables &amp;amp; Enable APIs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;1-set-up-environment-variables--enable-apis&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#1-set-up-environment-variables--enable-apis&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Define your operational variables and enable the required Google Cloud APIs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;$(&lt;/span&gt;gcloud config get-value project&lt;span class=&#34;k&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;europe-west3&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;destination-gcs-bucketname&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-to-gcs&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Enable required APIs for Cloud Run execution&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud services &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    run.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    cloudscheduler.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    secretmanager.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    iam.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    compute.googleapis.com&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;2-create-least-privilege-service-accounts&#34;&gt;(2) Create Least-Privilege Service Accounts&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;2-create-least-privilege-service-accounts&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#2-create-least-privilege-service-accounts&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Following least-privilege principles, create two dedicated service accounts: one for the Cloud Run job worker and one for the Cloud Scheduler cron trigger:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# 1. Create Service Account for the Cloud Run job&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts create sftp-run-sa &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --display-name&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Service Account for SFTP rclone Cloud Run job&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-run-sa@&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;.iam.gserviceaccount.com&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# 2. Create Service Account for Cloud Scheduler&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts create sftp-run-scheduler-sa &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --display-name&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Service Account for Cloud Scheduler to invoke Cloud Run&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;SCHEDULER_RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-run-scheduler-sa@&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;.iam.gserviceaccount.com&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;3-configure-gcs-bucket-permissions&#34;&gt;(3) Configure GCS Bucket Permissions&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;3-configure-gcs-bucket-permissions&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#3-configure-gcs-bucket-permissions&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Create your destination GCS bucket and grant the Cloud Run job service account object admin permissions:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the target GCS bucket if needed (Optional)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets create gs://&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --uniform-bucket-level-access
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Grant the Cloud Run SA permission to write to the GCS bucket&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets add-iam-policy-binding gs://&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/storage.objectAdmin&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;4-create-secrets-in-secret-manager&#34;&gt;(4) Create Secrets in Secret Manager&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;4-create-secrets-in-secret-manager&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#4-create-secrets-in-secret-manager&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;In Cloud Run jobs, configuration files and SSH keys are uploaded to Secret Manager and mounted directly as read-only volumes inside the container filesystem at runtime.&lt;/p&gt;
&lt;p&gt;Take the SSH private key file provided by the SFTP server administrator and save it to a local SSH private key file named &lt;code&gt;sftp_user_key&lt;/code&gt;. Next, upload this file as a secret:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the SSH key (Skip if already created in Batch recipe)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets create sftp-private-key &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --data-file&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp_user_key&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --replication-policy&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;automatic&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Next, generate a temporary &lt;code&gt;rclone.conf&lt;/code&gt; file for your environment (update &lt;code&gt;host&lt;/code&gt; and &lt;code&gt;user&lt;/code&gt; as applicable) and upload as a secret:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat &lt;span class=&#34;s&#34;&gt;&amp;lt;&amp;lt;EOF &amp;gt; rclone.conf
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;[gcs-ftp]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;type = sftp
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;host = sftp.example.com
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;user = gcs-ftp
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;key_file = /secrets/ssh/sftp_user_key
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;shell_type = cmd
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;md5sum_command = none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;sha1sum_command = none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;[gcs-bucket]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;type = google cloud storage
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;anonymous = false
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;bucket_policy_only = true
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;EOF&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Upload rclone.conf to Secret Manager and remove local copy&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets create rclone-config &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --data-file&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;rclone.conf&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --replication-policy&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;automatic&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;rm rclone.conf&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Grant the Cloud Run job worker service account permission to read both secrets:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets add-iam-policy-binding sftp-private-key &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/secretmanager.secretAccessor&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets add-iam-policy-binding rclone-config &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/secretmanager.secretAccessor&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;5-deploy-the-cloud-run-job&#34;&gt;(5) Deploy the Cloud Run job&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;5-deploy-the-cloud-run-job&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#5-deploy-the-cloud-run-job&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Next, we will create the Cloud Run job. First update the &lt;code&gt;SOURCE_PATH&lt;/code&gt; and &lt;code&gt;DESTINATION_PATH&lt;/code&gt; for your situation:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set source and destination paths (omit leading slashes unless required by your setup)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;SOURCE_PATH&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;source/path/here&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;DESTINATION_PATH&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;destination/path/here&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Next, we&amp;rsquo;ll create the job itself. The compute resources set here (&lt;code&gt;--cpu&lt;/code&gt; and &lt;code&gt;--memory&lt;/code&gt;) were enough to reach the 30 MiB/s networking limit when transferring 1 GiB files. If your transfer is limited by file processing and not throughput, adding resources could help. For jobs that are expected to take longer than an hour, increase the &lt;code&gt;--task-timeout&lt;/code&gt; accordingly:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Deploy the managed Cloud Run job container&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud run &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; create &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --image&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;docker.io/rclone/rclone:latest&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --region&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --service-account&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --cpu&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --memory&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;1Gi &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --tasks&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --max-retries&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --task-timeout&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;3600s &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --set-secrets&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;/secrets/config/rclone.conf=rclone-config:latest,/secrets/ssh/sftp_user_key=sftp-private-key:latest&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --args&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;copy,gcs-ftp:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;SOURCE_PATH&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;,gcs-bucket:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;DESTINATION_PATH&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;,--config=/secrets/config/rclone.conf,--verbose,--stats=60s,--stats-one-line-date,--stats-file-name-length=0,--transfers=8,--checkers=8,--log-file=/dev/stdout&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;6-test-job-execution--verify-logs&#34;&gt;(6) Test Job Execution &amp;amp; Verify Logs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;6-test-job-execution--verify-logs&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#6-test-job-execution--verify-logs&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Trigger an immediate execution:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Execute the job manually&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud run &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; execute &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --region&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;In Cloud Console navigate to the &lt;code&gt;Cloud Run&lt;/code&gt; -&amp;gt; &lt;code&gt;Jobs&lt;/code&gt; interface.&lt;/p&gt;
&lt;p&gt;View the job you created:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/cloud-run-1.png&#34; title=&#34;Cloud Run job page&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Run job page&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;View the specific execution details:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/cloud-run-2.png&#34; title=&#34;Cloud Run job execution details&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Run job execution details&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;View the resource usage including CPU and network for any bottlenecks:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/cloud-run-3.png&#34; title=&#34;Cloud Run job metrics&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Run job metrics&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;View the logging including each file copied by &lt;code&gt;rclone&lt;/code&gt;:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/cloud-run-4.png&#34; title=&#34;Cloud Run and rclone logs&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Run and rclone logs&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;7-scheduling-recurring-transfers-with-cloud-scheduler&#34;&gt;(7) Scheduling recurring transfers with Cloud Scheduler&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;7-scheduling-recurring-transfers-with-cloud-scheduler&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#7-scheduling-recurring-transfers-with-cloud-scheduler&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;To execute every two hours, configure Cloud Scheduler to trigger your job endpoint, and trigger it as a test:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Allow Cloud Scheduler to invoke the Cloud Run job&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud run &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --region&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;SCHEDULER_RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/run.invoker&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the scheduler job&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud scheduler &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; create http sftp-to-gcs-run-schedule &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --schedule&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;0 */2 * * *&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --uri&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;https://run.googleapis.com/v2/projects/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/locations/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/jobs/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;:run&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --http-method&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;POST &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --oauth-service-account-email&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;SCHEDULER_RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Run the schedule once manually to verify it works&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud scheduler &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; run sftp-to-gcs-run-schedule &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;If you check Cloud Scheduler you should see a successful invocation, and if you view the Cloud Run job page you can view the details of the &lt;code&gt;rclone&lt;/code&gt; transfer job execution.&lt;/p&gt;
&lt;p&gt;🥳 Congrats, you have now successfully used &lt;code&gt;rclone&lt;/code&gt; to transfer data from a remote SFTP server into GCS without a server to manage, and scheduled it to update every two hours!&lt;/p&gt;
&lt;hr&gt;
&lt;h2 data-hextra-search-id=&#34;recipe-2-batch&#34;&gt;Recipe 2: Batch&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;recipe-2-batch&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#recipe-2-batch&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Goal: Use Batch jobs to copy files from a remote SFTP server to a GCS bucket every two hours using a lightweight, zero-maintenance Batch job. These steps assume you are authenticated from a command prompt with &lt;code&gt;gcloud&lt;/code&gt; and a default project has been selected. Using &lt;code&gt;Cloud Shell&lt;/code&gt; is an option.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;1-set-up-environment-variables--enable-apis-1&#34;&gt;(1) Set Up Environment Variables &amp;amp; Enable APIs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;1-set-up-environment-variables--enable-apis-1&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#1-set-up-environment-variables--enable-apis-1&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Define your operational variables and enable the required Google Cloud APIs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;$(&lt;/span&gt;gcloud config get-value project&lt;span class=&#34;k&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;PROJECT_NUMBER&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;$(&lt;/span&gt;gcloud projects describe &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; --format&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;value(projectNumber)&amp;#34;&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;europe-west3&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;destination-gcs-bucketname&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;WORKFLOW_NAME&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-to-gcs-workflow&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud services &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    workflows.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    workflowexecutions.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    batch.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    compute.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    cloudscheduler.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    secretmanager.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    logging.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    iam.googleapis.com
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a default network if it doesn&amp;#39;t already exist&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud compute networks describe default &amp;gt;/dev/null 2&amp;gt;&lt;span class=&#34;p&#34;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;||&lt;/span&gt; gcloud compute networks create default --subnet-mode&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;auto&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;2-create-least-privilege-service-accounts--iam-bindings&#34;&gt;(2) Create Least-Privilege Service Accounts &amp;amp; IAM Bindings&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;2-create-least-privilege-service-accounts--iam-bindings&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#2-create-least-privilege-service-accounts--iam-bindings&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Following least-privilege principles, create three dedicated service accounts:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# 1. Create Service Account for the Batch worker VMs&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts create sftp-batch-sa &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --display-name&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Service Account for SFTP rclone Batch worker&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-batch-sa@&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;.iam.gserviceaccount.com&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# 2. Create Service Account for Workflow orchestrator&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts create sftp-workflows-sa &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --display-name&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Service Account for Workflow to submit Batch jobs&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;WORKFLOWS_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-workflows-sa@&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;.iam.gserviceaccount.com&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# 3. Create Service Account for Cloud Scheduler trigger&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts create sftp-batch-scheduler-sa &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --display-name&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Service Account for Cloud Scheduler to invoke Workflows&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;export&lt;/span&gt; &lt;span class=&#34;nv&#34;&gt;BATCH_SCHEDULER_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp-batch-scheduler-sa@&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;.iam.gserviceaccount.com&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Grant the necessary IAM roles for orchestration, token creation, and workflow execution:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Allow Workflows to create Batch jobs and act as the Batch worker SA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud projects add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;WORKFLOWS_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/batch.jobsEditor&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;WORKFLOWS_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/iam.serviceAccountUser&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Allow Cloud Scheduler service agent to mint tokens on behalf of BATCH_SCHEDULER_SA&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud iam service-accounts add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SCHEDULER_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:service-&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_NUMBER&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;@gcp-sa-cloudscheduler.iam.gserviceaccount.com&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/iam.serviceAccountTokenCreator&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Allow BATCH_SCHEDULER_SA to invoke Workflow&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud projects add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SCHEDULER_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/workflows.invoker&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;3-configure-gcs-bucket--batch-vm-permissions&#34;&gt;(3) Configure GCS Bucket &amp;amp; Batch VM Permissions&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;3-configure-gcs-bucket--batch-vm-permissions&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#3-configure-gcs-bucket--batch-vm-permissions&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Create the GCS bucket if needed, and then grant the Batch SA access to manage objects in it. Next, grant the Batch SA permission to report container health back to the Batch control plane and stream logs to Cloud Logging:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the target GCS bucket if needed (Optional)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets create gs://&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --uniform-bucket-level-access
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Grant the Batch worker SA permission to manage objects in the target GCS bucket&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets add-iam-policy-binding gs://&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/storage.objectAdmin&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Grant the Batch worker SA permission to report agent state back to the Batch control plane&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud projects add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/batch.agentReporter&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Grant the Batch worker SA permission to write logs to Cloud Logging&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud projects add-iam-policy-binding &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/logging.logWriter&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;4-create-secrets-in-secret-manager-1&#34;&gt;(4) Create Secrets in Secret Manager&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;4-create-secrets-in-secret-manager-1&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#4-create-secrets-in-secret-manager-1&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;With Batch, we will have the VM SA pull in the SSH private key at runtime. Take the SSH private key file provided by the SFTP server administrator and save it to a local SSH private key file named &lt;code&gt;sftp_user_key&lt;/code&gt;. Next, upload this file as a secret, and then grant the Batch SA permission to read it:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the SSH key (Skip if already created in Cloud Run jobs recipe)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets create sftp-private-key &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --data-file&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp_user_key&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --replication-policy&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;automatic&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Grant the Batch worker SA permission to access the SSH private key&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets add-iam-policy-binding sftp-private-key &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --member&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;serviceAccount:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --role&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;roles/secretmanager.secretAccessor&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;5-deploy-the-workflow-orchestrator&#34;&gt;(5) Deploy the Workflow Orchestrator&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;5-deploy-the-workflow-orchestrator&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#5-deploy-the-workflow-orchestrator&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Workflow provides a workflow you define in a YAML file. It is a quite expressive format that can be intimidating at first glance, but if you focus on a section at a time it gets easier.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The &lt;code&gt;init_config&lt;/code&gt; step includes variable assignments.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;submit_batch_transfer&lt;/code&gt; step creates the Batch job. Let&amp;rsquo;s step through the main parts:
&lt;ol&gt;
&lt;li&gt;The first runnable is &lt;code&gt;stage-config-and-keys&lt;/code&gt;, which puts the rclone config and SSH keys in place&lt;/li&gt;
&lt;li&gt;The second runnable is &lt;code&gt;run-rclone-sync&lt;/code&gt; and it defines the container details, similar to the Cloud Run recipe.&lt;/li&gt;
&lt;li&gt;You will also see &lt;code&gt;computeResource&lt;/code&gt; for CPU and Memory, and a &lt;code&gt;machineType&lt;/code&gt; for the VM shape to use. Batch uses cgroups to limit the task within the &lt;code&gt;computeResource&lt;/code&gt; limits so it is important to align these with the &lt;code&gt;machineType&lt;/code&gt; or you will potentially leave VM resources idle. For example if you wanted to use a 4 vCPU VM with 8 GiB of RAM you should set the &lt;code&gt;cpuMilli&lt;/code&gt; to &lt;code&gt;4000&lt;/code&gt; and &lt;code&gt;memoryMib&lt;/code&gt; to &lt;code&gt;8192&lt;/code&gt;. In my test environment transferring 32 x 1 GiB files I observed 2 vCPU at 300 MiB/s, 4 vCPU at 470 MiB/s, and 8 vCPU at 560 MiB/s. Experiment with different configurations to find the best fit for your needs.&lt;/li&gt;
&lt;li&gt;For jobs that are expected to take longer than an hour, increase the &lt;code&gt;maxRunDuration&lt;/code&gt; accordingly.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;First, set variables for your specific environment:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set SFTP host, SFTP user, and source and destination paths (omit leading slashes unless required by your setup)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;SFTP_HOST&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;sftp.example.com&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;SFTP_USER&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;gcs-ftp&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;SOURCE_PATH&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;source/path/here&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;DESTINATION_PATH&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;destination/path/here&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Next, create the Workflow YAML file:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat &lt;span class=&#34;s&#34;&gt;&amp;lt;&amp;lt;EOF &amp;gt; sftp-transfer-workflow.yaml
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;main:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;  steps:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;    - init_config:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;        assign:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - project_id: \${sys.get_env(&amp;#34;GOOGLE_CLOUD_PROJECT_ID&amp;#34;)}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - region: &amp;#34;${REGION}&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - unique_job_id: \${&amp;#34;sftp-sync-&amp;#34; + uuid.generate()}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - source_path: &amp;#34;${SOURCE_PATH}&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - destination_path: &amp;#34;${DESTINATION_PATH}&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          - rclone_config_content: |
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              [gcs-ftp]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              type = sftp
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              host = ${SFTP_HOST}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              user = ${SFTP_USER}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              key_file = /secrets/ssh/sftp_user_key
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              shell_type = cmd
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              md5sum_command = none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              sha1sum_command = none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              [gcs-bucket]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              type = google cloud storage
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              anonymous = false
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              bucket_policy_only = true
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;    - submit_batch_transfer:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;        call: googleapis.batch.v1.projects.locations.jobs.create
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;        args:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          parent: \${&amp;#34;projects/&amp;#34; + project_id + &amp;#34;/locations/&amp;#34; + region}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          jobId: \${unique_job_id}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;          body:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;            taskGroups:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              - taskCount: &amp;#34;1&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                parallelism: &amp;#34;1&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                taskSpec:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                  runnables:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    # Runnable 1: Stage our inline YAML config and fetch the secret SSH key to disk
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    - displayName: &amp;#34;stage-config-and-keys&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                      script:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                        text: |
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          mkdir -p /mnt/disks/secrets/ssh
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          echo &amp;#34;\${RCLONE_CONFIG_ENV}&amp;#34; &amp;gt; /mnt/disks/secrets/rclone.conf
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          unset CLOUDSDK_PYTHON
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          gcloud secrets versions access latest --secret=sftp-private-key --out-file=/mnt/disks/secrets/ssh/sftp_user_key
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          chmod 600 /mnt/disks/secrets/ssh/sftp_user_key
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          chmod 644 /mnt/disks/secrets/rclone.conf
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    # Runnable 2: Execute rclone transfer with clean standard output log redirection
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    - displayName: &amp;#34;run-rclone-sync&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                      container:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                        imageUri: &amp;#34;docker.io/rclone/rclone:latest&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                        commands:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;copy&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;gcs-ftp:${SOURCE_PATH}&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;gcs-bucket:${BUCKET_NAME}/${DESTINATION_PATH}&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--config=/secrets/rclone.conf&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--verbose&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--stats=60s&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--stats-one-line-date&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--stats-file-name-length=0&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--transfers=16&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--checkers=8&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;--log-file=/dev/stdout&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                        volumes:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                          - &amp;#34;/mnt/disks/secrets:/secrets&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                  environment:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    variables:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                      RCLONE_CONFIG_ENV: \${rclone_config_content}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                  computeResource:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    cpuMilli: &amp;#34;2000&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    memoryMib: &amp;#34;4096&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                  maxRetryCount: 2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                  maxRunDuration: &amp;#34;3600s&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;            allocationPolicy:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              serviceAccount:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                email: \${&amp;#34;sftp-batch-sa@&amp;#34; + project_id + &amp;#34;.iam.gserviceaccount.com&amp;#34;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              instances:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                - policy:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;                    machineType: &amp;#34;n4-highcpu-2&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;            logsPolicy:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;              destination: &amp;#34;CLOUD_LOGGING&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;        result: batch_execution_result
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;    - return_status:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;        return: \${batch_execution_result}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;EOF&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Deploy the workflow:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;gcloud workflows deploy ${WORKFLOW_NAME} \
    --source=sftp-transfer-workflow.yaml \
    --location=${REGION} \
    --service-account=${WORKFLOWS_SA_EMAIL}&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;6-submit--verify-job-execution&#34;&gt;(6) Submit &amp;amp; Verify Job Execution&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;6-submit--verify-job-execution&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#6-submit--verify-job-execution&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Execute the workflow to create the Batch job:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Trigger immediate execution of Workflow&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud workflows execute &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;WORKFLOW_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;In Cloud Console navigate to Batch. After a few seconds you should see the Batch job listed, and a bit later complete:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/batch-1.png&#34; title=&#34;Batch job page&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Batch job page&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;Click details, and then Logs, and view each file copied by &lt;code&gt;rclone&lt;/code&gt;:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/batch-2.png&#34; title=&#34;Batch job logs page&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Batch job logs page&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;Once &lt;code&gt;rclone&lt;/code&gt; finishes copying the files, Batch terminates and destroys the Compute Engine VM immediately.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;7-scheduling-recurring-transfers-with-cloud-scheduler-1&#34;&gt;(7) Scheduling recurring transfers with Cloud Scheduler&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;7-scheduling-recurring-transfers-with-cloud-scheduler-1&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#7-scheduling-recurring-transfers-with-cloud-scheduler-1&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;To execute on a recurring schedule every two hours, configure Cloud Scheduler to trigger your workflow endpoint:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create the scheduler job&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud scheduler &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; create http sftp-to-gcs-batch-schedule &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --schedule&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;0 */2 * * *&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --uri&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;https://workflowexecutions.googleapis.com/v1/projects/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;PROJECT_ID&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/locations/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/workflows/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;WORKFLOW_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/executions&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --http-method&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;POST &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --oauth-service-account-email&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BATCH_SCHEDULER_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --oauth-token-scope&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;https://www.googleapis.com/auth/cloud-platform&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Run the schedule once manually to verify it works&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud scheduler &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; run sftp-to-gcs-batch-schedule &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;If you check Cloud Scheduler you should see a successful invocation, and if you view the Batch job page you can view the details of the rclone transfer job execution.&lt;/p&gt;
&lt;p&gt;🥳 Congrats, you have now successfully used rclone to transfer data from a remote SFTP server into GCS without a server to manage, and scheduled it to update every two hours!&lt;/p&gt;
&lt;hr&gt;
&lt;h2 data-hextra-search-id=&#34;next-step--alternative-try-the-http-backend&#34;&gt;Next step / alternative: Try the HTTP backend&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;next-step--alternative-try-the-http-backend&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#next-step--alternative-try-the-http-backend&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;If you don&amp;rsquo;t have an external SFTP server but want to give &lt;code&gt;rclone&lt;/code&gt; with managed compute a try, here&amp;rsquo;s a fun way to do it using the &lt;code&gt;HTTP&lt;/code&gt; storage backend and a public dataset from the German weather service.  They maintain webcams at several stations and update still images every 10 minutes, retaining the last 48 hours of images. Let&amp;rsquo;s archive them into GCS!&lt;/p&gt;
&lt;p&gt;Append the following storage backend to your &lt;code&gt;rclone.conf&lt;/code&gt; file (tip: get your current file with &lt;code&gt;gcloud secrets versions access latest --secret=rclone-config --out-file=rclone.conf&lt;/code&gt;):&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat &lt;span class=&#34;s&#34;&gt;&amp;lt;&amp;lt;EOF &amp;gt;&amp;gt; rclone.conf
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;[dwd-de]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;type = http
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;url = https://opendata.dwd.de
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;EOF&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Create a new secret version:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud secrets versions add rclone-config &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --data-file&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;rclone.conf&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Set the path to use in your bucket and a name for this HTTP job:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Set destination paths (omit leading slashes unless required by your setup)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;DESTINATION_PATH&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;destination/path/here&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;http-webcam-to-gcs&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Deploy a &lt;strong&gt;Cloud Run&lt;/strong&gt; job:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud run &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; create &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --image&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;docker.io/rclone/rclone:latest&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --region&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --service-account&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;RUN_SA_EMAIL&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --cpu&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --memory&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;1Gi &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --tasks&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --set-secrets&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;/secrets/config/rclone.conf=rclone-config:latest&amp;#34;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --args&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;copy,dwd-de:weather/webcam,gcs-bucket:&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;BUCKET_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;DESTINATION_PATH&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;,--config=/secrets/config/rclone.conf,--include=*_full.jpg,--verbose,--stats=60s,--stats-one-line-date,--stats-file-name-length=0,--transfers=8,--checkers=8,--log-file=/dev/stdout&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Trigger an immediate execution:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud run &lt;span class=&#34;nb&#34;&gt;jobs&lt;/span&gt; execute &lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;JOB_NAME&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    --region&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;${&lt;/span&gt;&lt;span class=&#34;nv&#34;&gt;REGION&lt;/span&gt;&lt;span class=&#34;si&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;On each execution it will list files on the source and destination, and then download any files missing from the destination. If you plan to run this for the long term you will want to manage the destination directory structure to avoid unnecessary time and cost for large LIST operations as these images accumulate.&lt;/p&gt;
&lt;p&gt;But for now, enjoy views of the Port of Hamburg and the Elbe River:
&lt;img src=&#34;https://www.beginswithdata.com/post/2026-07-28/hamburg-1.jpg&#34; alt=&#34;image&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;conclusion&#34;&gt;Conclusion&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;conclusion&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#conclusion&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Transferring into and out of Google Cloud Storage using &lt;code&gt;rclone&lt;/code&gt; has never been easier! By using managed services like Cloud Run jobs, Batch, Workflow, Cloud Scheduler, and Secret Manager, you eliminate the hassle of running and paying for transfer VMs, while retaining the complete versatility of &lt;code&gt;rclone&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Moving GCS Buckets with Bucket Relocation</title>
      <link>https://www.beginswithdata.com/2025/10/14/gcs-buckets-relocate/</link>
      <pubDate>Tue, 14 Oct 2025 08:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2025/10/14/gcs-buckets-relocate/</guid>
      <description>
        
        
        &lt;p&gt;If you&amp;rsquo;ve ever had to move a Google Cloud Storage bucket from one storage location to another you know it’s not easy or cheap. Maybe you needed to move it to a different region to be closer to your compute instances for better performance, or perhaps to reduce cost. In the past, this meant a manual, multi-step process of creating a new bucket, transferring the data, and then updating all your applications to point to the new bucket. Retaining the same bucket name required an additional copy process back to the original bucket name. All this was time-consuming, error prone, and required a fair amount of downtime.&lt;/p&gt;
&lt;p&gt;Fortunately, GCS recently released the &lt;a href=&#34;https://cloud.google.com/storage/docs/bucket-relocation/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bucket Relocation&lt;/a&gt; feature that makes it much easier. In this post, you’ll learn the what, why, and how of this feature adding a new tool to your data management toolbelt.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;what-is-gcs-bucket-relocation&#34;&gt;What is GCS Bucket Relocation?&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;what-is-gcs-bucket-relocation&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#what-is-gcs-bucket-relocation&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Bucket Relocation, a feature of the &lt;a href=&#34;https://cloud.google.com/storage/docs/storage-intelligence/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Storage Intelligence&lt;/a&gt; product, allows you to &lt;strong&gt;change the storage location of an existing bucket from one location to another&lt;/strong&gt; with no downtime for reads, and limited or no downtime for writes.&lt;/p&gt;
&lt;p&gt;Some key points:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Easy to use&lt;/strong&gt;: No complex scripting or manual data transfers, as little as ONE command!&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Minimal Downtime&lt;/strong&gt;: Your applications can continue to read from the bucket during all phases of relocation. In the finalize phase writes are allowed or blocked depending on the location specifics of the move.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Metadata Preservation&lt;/strong&gt;: All object metadata, such as creation time, storage class, and custom metadata, are preserved during the move.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Friendly with Colder Storage Classes&lt;/strong&gt;: There are no retrieval fees for data stored at colder classes and Autoclass data is not promoted to Standard on transfer.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;No Application Changes&lt;/strong&gt;: Since the bucket name doesn&amp;rsquo;t change no client updates are needed.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Although there is one bucket relocation feature, under the covers there are two scenarios that come with different limitations:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Without write downtime&lt;/strong&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;With write downtime&lt;/strong&gt;&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Storage Locations&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Moves within the same continent (e.g. us, eu) between configurable dual region locations and/or a multi-region location. For example a configurable dual region us-east1/us-central1 to US multi-region.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;All other moves, for example across continents or when using regional or predefined dual-regions buckets. For example a regional bucket move from us-east1 to us-central1&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Finalize Stage&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No finalize stage required. After started the job will move data and manage the cutover completing after a 7-day period to ensure write consistency.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes, a finalize is required. During the finalize the bucket is read-only and all modifications such as writes or deletes will be blocked.&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;A small, but important list of &lt;a href=&#34;https://cloud.google.com/storage/docs/bucket-relocation/overview#relocation_without_write_downtime_limitations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;limitations&lt;/a&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;A longer, important and quite nuanced list of &lt;a href=&#34;https://cloud.google.com/storage/docs/bucket-relocation/overview#relocation_with_write_downtime_limitations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;limitations&lt;/a&gt;&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;h2 data-hextra-search-id=&#34;when-to-use-bucket-relocation&#34;&gt;When to use bucket relocation?&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;when-to-use-bucket-relocation&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#when-to-use-bucket-relocation&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;This is not a feature you set out to use on day one, but rather a feature that comes in extremely handy to respond to changes such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Improve Performance&lt;/strong&gt;: Co-locate storage and compute to reduce access latency and increase throughput.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Reduce Cost&lt;/strong&gt;: Co-locate storage with compute to reduce network charges. Move data archives to regions with lower pricing.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Change Resilience&lt;/strong&gt;: Move from regional to dual-regional, or multi-regional to regional, as your needs change.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 data-hextra-search-id=&#34;how-does-bucket-relocation-work&#34;&gt;How does bucket relocation work?&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;how-does-bucket-relocation-work&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#how-does-bucket-relocation-work&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;While the concept is simple - move a bucket from A to B - there are several stages to make it happen:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/how-it-works.png&#34; alt=&#34;Diagram of bucket relocate stages&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;You can use the CLI/API or Cloud Console to manage bucket relocation. Below you will find two sections with steps for the CLI and Cloud Console respectively.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;turning-theory-into-practice-cli&#34;&gt;Turning Theory into Practice (CLI)&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;turning-theory-into-practice-cli&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#turning-theory-into-practice-cli&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;I ran a bucket relocate job on a regional bucket (100k objects, 155 GiB) moving it from &lt;code&gt;europe-west4&lt;/code&gt; to &lt;code&gt;europe-west1&lt;/code&gt;. Here are the steps and my experience:&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;1-verify-prerequisites&#34;&gt;(1) Verify Prerequisites:&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;1-verify-prerequisites&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#1-verify-prerequisites&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Before you can relocate a bucket you must comply with the following three points:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Storage Intelligence&lt;/strong&gt;: The bucket relocation feature is part of Storage Intelligence, so you&amp;rsquo;ll need to have it enabled. You can enable at the project, folder, or org levels, using the 30-day trial if you like.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;IAM Roles&lt;/strong&gt;: You&amp;rsquo;ll need the Storage Admin (roles/storage.admin) IAM role on the project.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Soft Delete&lt;/strong&gt;: Your bucket must have soft delete enabled with at least 7 days of retention.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 data-hextra-search-id=&#34;2-dry-run&#34;&gt;(2) Dry Run:&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;2-dry-run&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#2-dry-run&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;I started a job with the &lt;code&gt;--dry-run&lt;/code&gt; flag and checked the job for success.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets relocate gs://bucket-on-the-move --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;europe-west1 --dry-run
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage operations list projects/_/buckets/bucket-on-the-move &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;3-initial-data-copy&#34;&gt;(3) Initial Data Copy&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;3-initial-data-copy&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#3-initial-data-copy&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Now I ran the exact same command but omitting the &lt;code&gt;--dry-run&lt;/code&gt; flag. The command returns the operation ID that you can use to monitor.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets relocate gs://bucket-on-the-move --location&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;europe-west1 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;WARNING: The bucket gs://bucket-on-the-move is in EUROPE-WEST4.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;WARNING: 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;1. This move will involve write downtime.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2. In-flight resumable uploads not finished before the write downtime will be lost.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;3. Bucket tags added to the bucket will result in the relocation being canceled.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;4. Please ensure that you have sufficient quota in the destination before performing the relocation.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Please acknowledge that you&lt;span class=&#34;err&#34;&gt;&amp;#39;&lt;/span&gt;ve &lt;span class=&#34;nb&#34;&gt;read&lt;/span&gt; the above warnings and want to relocate the bucket gs://bucket-on-the-move? &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;Y/n&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;?
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;...
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;selfLink: https://www.googleapis.com/storage/v1/b/bucket-on-the-move/operations/CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;4-monitor-job&#34;&gt;(4) Monitor Job&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;4-monitor-job&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#4-monitor-job&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Job statistics are available via the API or &lt;code&gt;gcloud&lt;/code&gt; but are not shown in an easy-to-read format and there is no &lt;code&gt;watch&lt;/code&gt; command. Also, they seem to be slow moving with infrequent updates; tens of minutes with no updated statistics was not unusual.&lt;/p&gt;
&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; fill=&#34;currentColor&#34; viewBox=&#34;3 3 18 18&#34;&gt;
  &lt;path d=&#34;M12 3C7.0275 3 3 7.12937 3 12.2276C3 16.3109 5.57625 19.7597 9.15374 20.9824C9.60374 21.0631 9.77249 20.7863 9.77249 20.5441C9.77249 20.3249 9.76125 19.5982 9.76125 18.8254C7.5 19.2522 6.915 18.2602 6.735 17.7412C6.63375 17.4759 6.19499 16.6569 5.8125 16.4378C5.4975 16.2647 5.0475 15.838 5.80124 15.8264C6.51 15.8149 7.01625 16.4954 7.18499 16.7723C7.99499 18.1679 9.28875 17.7758 9.80625 17.5335C9.885 16.9337 10.1212 16.53 10.38 16.2993C8.3775 16.0687 6.285 15.2728 6.285 11.7432C6.285 10.7397 6.63375 9.9092 7.20749 9.26326C7.1175 9.03257 6.8025 8.08674 7.2975 6.81794C7.2975 6.81794 8.05125 6.57571 9.77249 7.76377C10.4925 7.55615 11.2575 7.45234 12.0225 7.45234C12.7875 7.45234 13.5525 7.55615 14.2725 7.76377C15.9937 6.56418 16.7475 6.81794 16.7475 6.81794C17.2424 8.08674 16.9275 9.03257 16.8375 9.26326C17.4113 9.9092 17.76 10.7281 17.76 11.7432C17.76 15.2843 15.6563 16.0687 13.6537 16.2993C13.98 16.5877 14.2613 17.1414 14.2613 18.0065C14.2613 19.2407 14.25 20.2326 14.25 20.5441C14.25 20.7863 14.4188 21.0746 14.8688 20.9824C16.6554 20.364 18.2079 19.1866 19.3078 17.6162C20.4077 16.0457 20.9995 14.1611 21 12.2276C21 7.12937 16.9725 3 12 3Z&#34;&gt;&lt;/path&gt;
&lt;/svg&gt;
&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;I wrote a small shell script to monitor the relocation job progress, available in this GitHub repository:&lt;br&gt;
&lt;strong&gt;&lt;a href=&#34;https://github.com/dutchiechris/storage/tree/main/gcs/gcs-relocate-monitor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github.com/dutchiechris/storage/tree/main/gcs/gcs-relocate-monitor&lt;/a&gt;&lt;/strong&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;The output (from the whole job, start to finish, with largely redundant lines removed) was:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;./monitor-bucket-relocate.sh bucket-on-the-move CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Monitoring GCS operation: projects/_/buckets/bucket-on-the-move/operations/CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Polling every &lt;span class=&#34;m&#34;&gt;5&lt;/span&gt; seconds and output only on changes  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;________________________________________________________________________________________________________________________
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Timestamp &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;Elapsed&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;                     &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;Status                   &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;Job  &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;Objects                 &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;Capacity GiB            
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                                        &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;                         &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;%    &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;%    &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;done&lt;/span&gt;/total&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;       &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;%    &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;done&lt;/span&gt;/total&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;       
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 10:31:22 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;00:03:04&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;WAITING_ON_SYNC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;SYNCING&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;0/0&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;              &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;0.00/0.00&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 10:47:08 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;00:18:08&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;READY &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;SYNCING&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;          &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;0/100000&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;         &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;0%   &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;0.00/155.83&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 10:57:12 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;00:28:12&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;READY &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;SYNCING&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;          &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;41%  &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;41%  &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;41000/100000&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;     &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;41%  &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;63.89/155.83&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 11:02:10 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;00:34:50&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;READY &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;SYNCING&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;          &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;99%  &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;100000/100000&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;155.83/155.83&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 11:03:25 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;00:36:06&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;RUNNING &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;FINALIZING&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;     &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;99%  &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;100000/100000&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;155.83/155.83&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2025-10-10 11:59:23 UTC &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;01:31:59&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;      &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;FINALIZED &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;SUCCEEDED&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;100000/100000&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;100% &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;155.83/155.83&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;    
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;________________________________________________________________________________________________________________________
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Operation completed successfully.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;To view the raw output of this operation, run:
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage operations describe projects/_/buckets/bucket-on-the-move/operations/CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;5-final-synchronization&#34;&gt;(5) Final Synchronization&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;5-final-synchronization&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#5-final-synchronization&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Once I saw the Job status had reached 99% in the above output (after 36 minutes) I initiated the finalize:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets relocate --operation&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;projects/_/buckets/bucket-on-the-move/operations/CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ --finalize 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;WARNING: 
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;1. Any ongoing, in-flight resumable uploads will be canceled and lost.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2. Write downtime will be incurred.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;This will start the write downtime &lt;span class=&#34;k&#34;&gt;for&lt;/span&gt; your relocation of gs://bucket-on-the-move, are you sure you want to
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt; &lt;span class=&#34;k&#34;&gt;continue&lt;/span&gt;? &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;Y/n&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;?  Y
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Sent request to advance relocation &lt;span class=&#34;k&#34;&gt;for&lt;/span&gt; bucket gs://bucket-on-the-move with operation CiQxOTI1OWYyOS00ZjBmLTRjNzEtYjAzMC0wMjZlMGI5ZWI1MDUQBQ.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;I then tried to write an object to the bucket and observed the write lock blocked it as expected:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage cp file.data gs://bucket-on-the-move/updates/file.data.new
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Copying file://file.data to gs://bucket-on-the-move/updates/file.data.new
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;ERROR: HTTPError 412: This bucket is being relocated and is in the finalization step. Object writes are blocked during this state.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;As expected, reads are still allowed:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage cp gs://bucket-on-the-move/updates/file.data file.data
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Copying gs://bucket-on-the-move/updates/file.data to file://file.data
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  Completed files 1/1 &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; 83.8kiB/83.8kiB &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;6-validation-and-post-relocation-tasks&#34;&gt;(6) Validation and Post Relocation Tasks&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;6-validation-and-post-relocation-tasks&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#6-validation-and-post-relocation-tasks&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;After an hour or so I saw my monitor script completed reporting a successful bucket move. I checked the bucket location saw it was updated, and then copied a file to verify the object write lock had been removed:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-shell&#34; data-lang=&#34;shell&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage buckets describe gs://bucket-on-the-move &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; grep location:
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;location: EUROPE-WEST1
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;gcloud storage cp file.data gs://bucket-on-the-move/updates/file.data.new
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Copying file://file.data to gs://bucket-on-the-move/updates/file.data.new
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  Completed files 1/1 &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; 83.8kiB/83.8kiB    &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;I wasn’t using object tags or inventory reports, and don’t have an IaaC like Terraform to update a bucket location, so all tasks are done. 🎉&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;turning-theory-into-practice-cloud-console&#34;&gt;Turning Theory into Practice (Cloud Console)&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;turning-theory-into-practice-cloud-console&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#turning-theory-into-practice-cloud-console&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;You can also relocate a bucket from the Cloud Console. I ran a bucket relocate job on a regional bucket (2 objects, 33 MiB) moving it from &lt;code&gt;europe-west4&lt;/code&gt; to &lt;code&gt;europe-west1&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Click on a bucket, and then the &lt;code&gt;Configuration&lt;/code&gt; tab and you will see a Edit pencil icon next to the bucket location:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/edit-bucket.png&#34; alt=&#34;Edit bucket location&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Choose the desired destination location and start a dry-run:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/dry-run.png&#34; alt=&#34;Start dry-run&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Review the results, in this case there were several files which had been uploaded via XML Multipart Uploads (MPU) which are not supported. Details of the object names are in Cloud Logging:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/dry-run-results.png&#34; alt=&#34;Review results of dry-run&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Fix any issues, and then start the actual data sync using the &lt;code&gt;Relocate now&lt;/code&gt; button.&lt;/p&gt;
&lt;p&gt;During the initial sync the bucket remains in read/write mode:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/start-copy.png&#34; alt=&#34;Start sync&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;When the sync is 99% complete, click the &lt;code&gt;Start final synchronization&lt;/code&gt; button to start the cutover. For a region to region move the bucket will become read-only:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/final-sync.png&#34; alt=&#34;Start cutover&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;After some time, the sync will complete and the bucket will automatically become read/write again in the new storage location:
&lt;img src=&#34;https://www.beginswithdata.com/post/2025-10-14/relocate-complete.png&#34; alt=&#34;Cutover complete&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;gotchas-and-things-to-consider&#34;&gt;Gotchas and Things to Consider&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;gotchas-and-things-to-consider&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#gotchas-and-things-to-consider&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;While experimenting here are a few things I discovered to keep in mind:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;: Read the limitations (&lt;a href=&#34;https://cloud.google.com/storage/docs/bucket-relocation/overview#relocation_without_write_downtime_limitations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Without write downtime&lt;/a&gt; or &lt;a href=&#34;https://cloud.google.com/storage/docs/bucket-relocation/overview#relocation_with_write_downtime_limitations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;With write downtime&lt;/a&gt;) that are applicable to your scenario. &lt;b style=&#39;color:red;&#39;&gt;Don’t skim them, read them thoroughly.&lt;/b&gt;
The use of XML Multipart uploads (temporarily or ever depending on scenario), HNS buckets, having locked objects, etc, can all prevent the use of this feature. The limitations are  nuanced and rather than repeating here please read the docs directly. The &lt;code&gt;--dry-run&lt;/code&gt; is a safeguard to run before the move if you expect your bucket is supported for a move.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Cost&lt;/strong&gt;: You incur a relocation fee which includes a transfer charge of $0.04 per GiB of data moved, plus Class A operations charges for the relocation operation (billed per object moved), and potentially data transfer network fees and GCS inter-region replication fees if moving to a dual-region or multi-region.  Your bucket must also be enrolled in Storage Intelligence, but there is a 30 day free trial removing any cost if you unenroll after the bucket relocate completes.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Anywhere Cache&lt;/strong&gt;: If your data is in a multi-region bucket and you want to reduce network costs, considering adding an Anywhere Cache to the compute zones that access it. For some workloads this will save you the network costs AND increase performance and you can leave the bucket where it is!&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 data-hextra-search-id=&#34;conclusion&#34;&gt;Conclusion&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;conclusion&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#conclusion&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;The bucket relocation feature allows you to move buckets with relative ease to optimize cost, improve performance, or achieve resilience goals. Use the knowledge gained in this post to give it a try in your environment!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Using AWS S3 Tools with Google Cloud Storage: Critical Update</title>
      <link>https://www.beginswithdata.com/2025/05/14/aws-s3-tools-with-gcs/</link>
      <pubDate>Wed, 14 May 2025 00:00:00 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2025/05/14/aws-s3-tools-with-gcs/</guid>
      <description>
        
        
        &lt;h2 data-hextra-search-id=&#34;introduction&#34;&gt;&lt;strong&gt;Introduction&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;introduction&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#introduction&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Google Cloud Storage (GCS) is a service for storing objects in Google Cloud. While GCS provides native &lt;a href=&#34;https://cloud.google.com/storage/docs/xml-api/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;XML&lt;/a&gt;, &lt;a href=&#34;https://cloud.google.com/storage/docs/json_api&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JSON&lt;/a&gt;, and &lt;a href=&#34;https://cloud.google.com/storage/docs/enable-grpc-api&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;gRPC&lt;/a&gt; APIs, Google recommends using a higher level interface such as &lt;a href=&#34;https://cloud.google.com/storage/docs/cloud-storage-fuse/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;GCS Fuse&lt;/a&gt; or the &lt;a href=&#34;https://cloud.google.com/storage/docs/reference/libraries&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Cloud Storage client libraries&lt;/a&gt; when possible. Higher level interfaces simplify integration, provide optimal GCS performance, and are actively developed with new features and bug fixes.&lt;/p&gt;
&lt;p&gt;Many customers have existing software built for Amazon S3 using the &lt;a href=&#34;https://docs.aws.amazon.com/AmazonS3/latest/API/sdk-general-information-section.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AWS SDK&lt;/a&gt;. GCS&amp;rsquo;s XML interface is S3-compatible and often existing code needs only &lt;a href=&#34;https://cloud.google.com/storage/docs/aws-simple-migration&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;minor adjustments&lt;/a&gt; to work with GCS. However, don&amp;rsquo;t forget that AWS designs &lt;em&gt;their software&lt;/em&gt; for use with &lt;em&gt;their services&lt;/em&gt; and may make changes without considering other providers.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;the-breaking-change&#34;&gt;&lt;strong&gt;The Breaking Change&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;the-breaking-change&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#the-breaking-change&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;AWS recently added &lt;a href=&#34;https://aws.amazon.com/blogs/aws/introducing-default-data-integrity-protections-for-new-objects-in-amazon-s3/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;default data integrity protections to their SDKs and CLI&lt;/a&gt;. While beneficial for S3 users, this change made the default settings of the AWS SDK incompatible with most third-party S3-compatible services, including GCS.&lt;/p&gt;
&lt;p&gt;After upgrading your AWS SDK, some operations (including uploads and downloads) will fail against GCS with errors like:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: Invalid argument.
Expected checksum x4Vs7w== did not match calculated checksum: u4E0XQ==&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Though providers may eventually adapt their endpoints to be compatible with the new AWS tool defaults, in the meantime it&amp;rsquo;s possible revert to the previous behavior.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Impacted Versions&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Amazon rolled out this default incrementally. My testing shows the following version history for when the defaults changed and the ability to override became available:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Boto3 (Python SDK):&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;boto3 &amp;lt;= 1.35.99: No checksum config needed.&lt;/li&gt;
&lt;li&gt;boto3 &amp;gt;= 1.36.00: &lt;strong&gt;Requires&lt;/strong&gt; &lt;code&gt;when_required&lt;/code&gt; checksum config.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;AWS CLI:&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;awscli &amp;lt;= 2.22.35: No checksum config needed.&lt;/li&gt;
&lt;li&gt;awscli &amp;gt;= 2.23.0 &amp;lt;= 2.23.4: &lt;strong&gt;Incompatible&lt;/strong&gt;; checksum config unavailable.&lt;/li&gt;
&lt;li&gt;awscli &amp;gt;= 2.23.5: &lt;strong&gt;Requires&lt;/strong&gt; &lt;code&gt;when_required&lt;/code&gt; checksum config.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Other AWS SDKs are also affected. For example, see the &lt;a href=&#34;https://github.com/aws/aws-sdk-go-v2/discussions/2960&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;GitHub announcement&lt;/a&gt; for the Go SDK updates.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;the-solution&#34;&gt;&lt;strong&gt;The Solution&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;the-solution&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#the-solution&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;For the latest versions of AWS tools to work with GCS, two settings must be applied. These settings force the libraries to use the &lt;em&gt;&lt;strong&gt;previous default checksum parameters&lt;/strong&gt;&lt;/em&gt; that are compatible with GCS. They can be configured via environment variables (for both AWS CLI and Python SDK) or client configuration (e.g., Boto client config, AWS CLI profile).&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;environment-variables&#34;&gt;&lt;strong&gt;Environment Variables:&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;environment-variables&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#environment-variables&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Set these before running your application or CLI commands:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;AWS_REQUEST_CHECKSUM_CALCULATION=&amp;#39;when_required&amp;#39;
AWS_RESPONSE_CHECKSUM_VALIDATION=&amp;#39;when_required&amp;#39;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;client-configuration-boto-client-config-or-aws-cli-profile&#34;&gt;&lt;strong&gt;Client Configuration (Boto client config or AWS CLI profile):&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;client-configuration-boto-client-config-or-aws-cli-profile&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#client-configuration-boto-client-config-or-aws-cli-profile&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Alternatively, configure these directly:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;request_checksum_calculation=&amp;#39;when_required&amp;#39;
response_checksum_validation=&amp;#39;when_required&amp;#39;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h3 data-hextra-search-id=&#34;practical-examples&#34;&gt;&lt;strong&gt;Practical Examples:&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;practical-examples&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#practical-examples&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Python (Boto3):&lt;/strong&gt; See the &lt;a href=&#34;https://github.com/dutchiechris/storage/blob/main/gcs/gcs-aws-tools-compatibility/gcs-aws-boto3.py#L48&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;config syntax example&lt;/a&gt; and &lt;a href=&#34;https://github.com/dutchiechris/storage/blob/main/gcs/gcs-aws-tools-compatibility/gcs-aws-boto3.py#L79&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;environment variable technique&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;AWS CLI:&lt;/strong&gt; An example of the &lt;a href=&#34;https://github.com/dutchiechris/storage/blob/main/gcs/gcs-aws-tools-compatibility/gcs-aws-cli.sh#L61&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;environment variable technique&lt;/a&gt; with Docker is available.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Go:&lt;/strong&gt; A simple test using this &lt;a href=&#34;https://stackoverflow.com/a/79379236&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Stack Overflow solution&lt;/a&gt; worked for me.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; fill=&#34;currentColor&#34; viewBox=&#34;3 3 18 18&#34;&gt;
  &lt;path d=&#34;M12 3C7.0275 3 3 7.12937 3 12.2276C3 16.3109 5.57625 19.7597 9.15374 20.9824C9.60374 21.0631 9.77249 20.7863 9.77249 20.5441C9.77249 20.3249 9.76125 19.5982 9.76125 18.8254C7.5 19.2522 6.915 18.2602 6.735 17.7412C6.63375 17.4759 6.19499 16.6569 5.8125 16.4378C5.4975 16.2647 5.0475 15.838 5.80124 15.8264C6.51 15.8149 7.01625 16.4954 7.18499 16.7723C7.99499 18.1679 9.28875 17.7758 9.80625 17.5335C9.885 16.9337 10.1212 16.53 10.38 16.2993C8.3775 16.0687 6.285 15.2728 6.285 11.7432C6.285 10.7397 6.63375 9.9092 7.20749 9.26326C7.1175 9.03257 6.8025 8.08674 7.2975 6.81794C7.2975 6.81794 8.05125 6.57571 9.77249 7.76377C10.4925 7.55615 11.2575 7.45234 12.0225 7.45234C12.7875 7.45234 13.5525 7.55615 14.2725 7.76377C15.9937 6.56418 16.7475 6.81794 16.7475 6.81794C17.2424 8.08674 16.9275 9.03257 16.8375 9.26326C17.4113 9.9092 17.76 10.7281 17.76 11.7432C17.76 15.2843 15.6563 16.0687 13.6537 16.2993C13.98 16.5877 14.2613 17.1414 14.2613 18.0065C14.2613 19.2407 14.25 20.2326 14.25 20.5441C14.25 20.7863 14.4188 21.0746 14.8688 20.9824C16.6554 20.364 18.2079 19.1866 19.3078 17.6162C20.4077 16.0457 20.9995 14.1611 21 12.2276C21 7.12937 16.9725 3 12 3Z&#34;&gt;&lt;/path&gt;
&lt;/svg&gt;
&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;For more code examples, visit this GitHub repository:&lt;br&gt;
&lt;strong&gt;&lt;a href=&#34;https://github.com/dutchiechris/storage/tree/main/gcs/gcs-aws-tools-compatibility&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github.com/dutchiechris/storage/tree/main/gcs/gcs-aws-tools-compatibility&lt;/a&gt;&lt;/strong&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;h2 data-hextra-search-id=&#34;conclusion&#34;&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;conclusion&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#conclusion&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;While using AWS SDKs with GCS can ease and accelerate adoption, be sure to test compatibility before adopting new client versions. For this latest AWS update applying these simple changes should regain compatibility that was lost due to AWS’s recent updates to default data integrity protection.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Faster and Cheaper? Google Cloud&#39;s N2 vs C4</title>
      <link>https://www.beginswithdata.com/2025/04/28/n2-vs-c4-showdown/</link>
      <pubDate>Mon, 28 Apr 2025 00:00:00 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2025/04/28/n2-vs-c4-showdown/</guid>
      <description>
        
        
        &lt;p&gt;If you’ve been deploying N2 VMs on Google Cloud for the last few years it’s time to make a change. The latest generation 3 and 4 machine series deliver more performance at lower cost. How they do it isn&amp;rsquo;t one thing but a mix including more powerful CPUs, &lt;a href=&#34;https://cloud.google.com/titanium&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Titanium infrastructure&lt;/a&gt; enhancements including Hyperdisk and offload adapters, updated hypervisor software and NUMA aligned scheduling logic, and more. The tech is amazing, but most folks just care about what it means for their workloads. I recently built a couple of demos for Google Cloud NEXT to showcase the benefits for two scenarios: Databases and Software Development. Check ‘em out!&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;databases&#34;&gt;Databases&lt;/h1&gt;&lt;p&gt;Imagine you need a MySQL database for an OLTP workload of 70k DML operations per second with about 90 GiB of data. Lets map those requirements to both an N2 and a C4 machine types, with a VM shape loaded to no more than 70% utilization. Doing that I discovered that with C4 the the cost dropped by 20% and 95th percentile latency improved by more than 4x! Cheaper and faster, that’s my kind of combo.&lt;/p&gt;
&lt;iframe width=&#34;560&#34; height=&#34;315&#34; src=&#34;https://www.youtube.com/embed/FapVp8RySR8?si=hmNG9nIiL20Ia2fR&#34; title=&#34;YouTube video player&#34; frameborder=&#34;0&#34; allow=&#34;accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share&#34; referrerpolicy=&#34;strict-origin-when-cross-origin&#34; allowfullscreen&gt;&lt;/iframe&gt;
&lt;h1 data-hextra-search-id=&#34;software-development&#34;&gt;Software Development&lt;/h1&gt;&lt;p&gt;Build jobs are often CPU intensive and building the open source Chromium browser is no different. With over 50,000 build steps of mostly C++ code compilation it can take a few hours to complete. For this one I ran it on both an N2 and C4 standard-16 shapes and found that on the C4 it finished in 25% less time and 25% lower cost. Cheaper and faster, again, music to my ears.&lt;/p&gt;
&lt;iframe width=&#34;560&#34; height=&#34;315&#34; src=&#34;https://www.youtube.com/embed/ViAwp4YTla0?si=EoPYzIMILlrRJQEp&#34; title=&#34;YouTube video player&#34; frameborder=&#34;0&#34; allow=&#34;accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share&#34; referrerpolicy=&#34;strict-origin-when-cross-origin&#34; allowfullscreen&gt;&lt;/iframe&gt;
&lt;p&gt;So what are you waiting for? For new workloads, or to give a bit more power to your existing ones, try the latest generation VM series available now on Google Cloud!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Restricting GCS buckets by IP address</title>
      <link>https://www.beginswithdata.com/2024/12/02/gcs-buckets-ip-filter/</link>
      <pubDate>Mon, 02 Dec 2024 00:00:00 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2024/12/02/gcs-buckets-ip-filter/</guid>
      <description>
        
        
        &lt;p&gt;A Google Cloud Storage (GCS) bucket is accessible to all internet and Virtual Private Cloud (VPC) network client requests that include valid credentials. To further restrict access based on source IP addresses configure the &lt;a href=&#34;https://cloud.google.com/storage/docs/ip-filtering-overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bucket IP filtering&lt;/a&gt; feature. When enabled, only requests from permitted IPv4 and/or IPv6 address ranges for internet and VPC traffic are allowed, and all other requests are denied.&lt;/p&gt;
&lt;p&gt;In this blog I will demonstrate how to use bucket IP filtering feature and recover from a situation where you have been locked out. For simplicity I will set the bucket to be public, but the IP filtering concepts are the same for a non-public bucket. I assume you already have a test project with billing enabled and have enabled the Compute Engine API using the default VPC network.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;define-variables&#34;&gt;Define variables&lt;/h1&gt;&lt;p&gt;We will use the Cloud Shell for many tasks. For ease of use define some helper variables. Note, if your Cloud Console session gets disconnected you will need to run these again after reconnecting. &lt;strong&gt;TIP:&lt;/strong&gt; start two Cloud Shell tabs and in the 1st one run the &lt;code&gt;top&lt;/code&gt; command, and then use the 2nd one for the commands in this blog. Running &lt;code&gt;top&lt;/code&gt; in the first tab seems to prevent the shell from disconnecting. 👌&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Open Cloud Shell from the Google Cloud Console.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set variables for your environment including project, region, and bucket names accordingly:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PROJECT=gcs-20241202
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;REGION=europe-west1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;ZONE=$REGION-b
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;BUCKET=ipfilter-20241202
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;GCPUSER=$(gcloud auth list --filter=status:ACTIVE --format=&amp;#34;value(account)&amp;#34;)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set the default &lt;code&gt;gcloud&lt;/code&gt; project so we don’t need to supply it on each command:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud config set project $PROJECT
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 data-hextra-search-id=&#34;create-and-test-with-a-public-bucket&#34;&gt;Create and test with a public bucket&lt;/h1&gt;&lt;p&gt;In the first set of steps create a bucket, set it public, and upload a test file. This is the default behaviour of GCS where there are no network client IP address restrictions resulting in this configuration:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-12-02/1.png&#34; title=&#34;Public Bucket&#34; alt=&#34;Public Bucket&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Public Bucket&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;create-bucket-make-it-public-and-upload-a-test-object&#34;&gt;Create bucket, make it public, and upload a test object&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;create-bucket-make-it-public-and-upload-a-test-object&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#create-bucket-make-it-public-and-upload-a-test-object&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;From the Cloud Shell create a new bucket and upload a test object.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Create a bucket.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud storage buckets create gs://$BUCKET --location=$REGION 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Make the bucket public:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud storage buckets add-iam-policy-binding gs://$BUCKET --member=allUsers --role=roles/storage.objectViewer 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a test object:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cat &amp;lt;&amp;lt; EOF &amp;gt; hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Hello World!
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Upload the test object to the bucket:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud storage cp hello.txt gs://$BUCKET --cache-control=no-store
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 data-hextra-search-id=&#34;test-access-from-the-internet&#34;&gt;Test access from the Internet&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;test-access-from-the-internet&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#test-access-from-the-internet&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Verify access to the object from various network locations.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Print out access commands / URLs so we can verify access from other network client locations:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo -e &amp;#34;\n&amp;#34; 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from terminal: &amp;#34; curl -s https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://tools.keycdn.com/performance?url=https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Cut-n-paste the first output above and verify access from Cloud Shell. Next, click the second from the output above and observe the file can be opened from your web browser. Lastly, click the third from the output above, then the blue &lt;code&gt;Test&lt;/code&gt; button, and you should see the file is accessed from a variety of locations on the Internet with a 200 OK HTTP Status code.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;test-access-from-your-vpc&#34;&gt;Test access from your VPC&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;test-access-from-your-vpc&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#test-access-from-your-vpc&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Create two test VMs (&lt;code&gt;node-allow&lt;/code&gt; and &lt;code&gt;node-deny&lt;/code&gt;) in your VPC and verify the object can be accessed.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;From Cloud Shell, create two VMs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;for ID in allow deny; do
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances create node-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--machine-type=e2-micro \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--shielded-secure-boot \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--network-interface=stack-type=IPV4_ONLY,subnet=default \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--zone=$REGION-b \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--scopes=https://www.googleapis.com/auth/devstorage.full_control,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/trace.append \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--create-disk=auto-delete=yes,boot=yes,mode=rw,size=10,image-project=ubuntu-os-cloud,image-family=ubuntu-2404-lts-amd64,type=pd-standard
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;done
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Allow ingress traffic from all IP addresses that IAP uses for TCP forwarding.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute firewall-rules create allow-ssh-ingress-from-iap \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --direction=INGRESS \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --action=allow \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --rules=tcp:22 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --source-ranges=35.235.240.0/20
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;In Cloud Console, click the Connect SSH to open new terminal sessions to each VM.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;From each VM, run the &lt;code&gt;Use from terminal&lt;/code&gt; curl command from above to verify the file can be fetched from each VM.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We&amp;rsquo;ve verified that the object can be read from Internet and VPC clients. Now we can configure Bucket IP filtering!&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;apply-bucket-ip-filter-for-an-internet-network-client&#34;&gt;Apply Bucket IP Filter for an Internet network client&lt;/h1&gt;&lt;p&gt;Set a Bucket IP Filter and allow a single host resulting in this configuration:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-12-02/2.png&#34; title=&#34;Internet IP Filtered Bucket&#34; alt=&#34;Internet IP Filtered Bucket&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Internet IP Filtered Bucket&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;configure-permissions-and-roles&#34;&gt;Configure permissions and roles&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;configure-permissions-and-roles&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#configure-permissions-and-roles&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Bucket IP filtering introduces several new IAM permissions that are not included in the existing default roles. We need to add and bind these to our user, and then create and apply the IP bucket filter itself to allow only our browser&amp;rsquo;s IP address.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;From Cloud Shell, create a new role that allows management of Bucket IP filtering&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud iam roles create gcs_ipfilter \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --title=&amp;#34;GCS Bucket IP Filter Management Role&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --description=&amp;#34;Allows setting IP filters on buckets&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --permissions=storage.buckets.setIpFilter,storage.buckets.getIpFilter \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --stage=GA
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Bind the Bucket IP filtering management role to your user:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud projects add-iam-policy-binding $PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --member=&amp;#34;user:$GCPUSER&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --role=&amp;#34;projects/$PROJECT/roles/gcs_ipfilter&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 data-hextra-search-id=&#34;configure-bucket-ip-filter-on-the-internet-ip&#34;&gt;Configure Bucket IP Filter on the Internet IP&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;configure-bucket-ip-filter-on-the-internet-ip&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#configure-bucket-ip-filter-on-the-internet-ip&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Get the Internet IP address used by your web browser, for example by visiting &lt;a href=&#34;https://ifconfig.me/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ifconfig.me&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;From Cloud Shell, set it as a variable:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;CLIENT_IP=&amp;lt;your browser IP&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create your Bucket IP filter config; check the &lt;a href=&#34;https://cloud.google.com/storage/docs/create-ip-filter#ip-filtering-configurations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;docs&lt;/a&gt; for full syntax:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cat &amp;lt;&amp;lt; EOF &amp;gt; allow-client-ip.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;{
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  &amp;#34;mode&amp;#34;: &amp;#34;Enabled&amp;#34;,
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  &amp;#34;publicNetworkSource&amp;#34;:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    &amp;#34;allowedIpCidrRanges&amp;#34;: [&amp;#34;$CLIENT_IP/32&amp;#34;]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    }
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Apply IP Bucket filter to our bucket:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud alpha storage buckets update gs://$BUCKET --ip-filter-file=allow-client-ip.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Using Cloud Shell, show the commands to check reachability:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo -e &amp;#34;\n&amp;#34; 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from terminal: &amp;#34; curl -s https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://tools.keycdn.com/performance?url=https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;You should see that the file is still accessible from your browser, but is no longer accessible from either the terminal or the CDN testing site. Additionally, from the terminal (Cloud Shell or your VMs) you&amp;rsquo;ll see the error &lt;code&gt;There is an IP filtering condition that is preventing access to the resource.&lt;/code&gt; Because the Internet IP address of your web browser is allowed you will still be able to manage the bucket from the Clone Console UI.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;allow-access-to-an-internet-network-and-vpc-network-ip-address&#34;&gt;Allow access to an Internet network and VPC network IP address&lt;/h1&gt;&lt;p&gt;Expand access to allow one of our VPC VM IP Addresses resulting in this configuration:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-12-02/3.png&#34; title=&#34;Internet and VPC Network IP Filtered Bucket&#34; alt=&#34;Internet and VPC Network IP Filtered Bucket&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Internet and VPC Network IP Filtered Bucket&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;expand-bucket-ip-filter-to-one-of-the-vpc-vms&#34;&gt;Expand Bucket IP Filter to one of the VPC VMs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;expand-bucket-ip-filter-to-one-of-the-vpc-vms&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#expand-bucket-ip-filter-to-one-of-the-vpc-vms&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;When a Bucket IP filter is applied all access is blocked for Internet and VPC clients unless expressly permitted. Let&amp;rsquo;s add one of our VMs on the VPC for access.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Using Cloud Shell get the IP address of the &lt;code&gt;node-allow&lt;/code&gt; VM and save it to a variable:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;ALLOWIP=$(gcloud compute instances describe node-allow --zone=$REGION-b --format=&amp;#39;value(networkInterfaces[0].accessConfigs[0].natIP)&amp;#39;)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a new Bucket IP filter file which includes both the Internet IP and the VPC network &lt;code&gt;node-allow&lt;/code&gt; VM; check the &lt;a href=&#34;https://cloud.google.com/storage/docs/create-ip-filter#ip-filtering-configurations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;docs&lt;/a&gt; for full syntax:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cat &amp;lt;&amp;lt; EOF &amp;gt; allow-client-and-vpc-ip.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;{
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  &amp;#34;mode&amp;#34;: &amp;#34;Enabled&amp;#34;,
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  &amp;#34;publicNetworkSource&amp;#34;:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     &amp;#34;allowedIpCidrRanges&amp;#34;: [&amp;#34;$CLIENT_IP/32&amp;#34;]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    },
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  &amp;#34;vpcNetworkSources&amp;#34;:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    [
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     {&amp;#34;network&amp;#34;: &amp;#34;projects/$PROJECT/global/networks/default&amp;#34;,  &amp;#34;allowedIpCidrRanges&amp;#34; : [&amp;#34;$ALLOWIP/32&amp;#34;]}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    ]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Apply it:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud alpha storage buckets update gs://$BUCKET --ip-filter-file=allow-client-and-vpc-ip.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Updating gs://ipfilter-20241202/...                                                             
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;ERROR: [chris@myorg.com] does not have permission to access b instance [ipfilter-20241202] (or it may not exist): There is an IP filtering
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;condition that is preventing access to the resource. This command is authenticated as chris@myorg.com which is the active account specified
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;by the [core/account] property.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  Completed 0  
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Oops. While you can manage the bucket from your browser, Cloud Shell runs in Google Cloud on a VPC Network and is locked out. There are two options: (a) update the IP filter from the allowed Internet IP or (b) grant the user permission to be &lt;a href=&#34;https://cloud.google.com/storage/docs/bypass-ip-filter&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;exempted&lt;/a&gt; from the IP filter &lt;em&gt;for bucket configuration changes&lt;/em&gt;. To demonstrate the exempt scenario we&amp;rsquo;ll go with option (b).&lt;/p&gt;
&lt;p&gt;In production it would be preferred to manage Bucket IP Filter from a well known client instead of Cloud Shell. Also, if exempting a user consider using &lt;a href=&#34;https://cloud.google.com/iam/docs/pam-overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Privileged Access Manager (PAM)&lt;/a&gt; to control just-in-time temporary privilege elevation for select principals, and to view audit logs afterwards to find out who had access to what and when.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;grant-bucket-ip-filter-exempt-role-and-expand-access-to-one-of-the-vpc-vms&#34;&gt;Grant Bucket IP Filter exempt role and expand access to one of the VPC VMs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;grant-bucket-ip-filter-exempt-role-and-expand-access-to-one-of-the-vpc-vms&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#grant-bucket-ip-filter-exempt-role-and-expand-access-to-one-of-the-vpc-vms&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;From Cloud Shell, create a Bucket IP Filter exempt role&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud iam roles create gcs_ipfilter_exempt \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --title=&amp;#34;GCS Bucket IP Filter Exempt Management Role&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --description=&amp;#34;Allows configuration overrides regardless of IP filters on buckets&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --permissions=storage.buckets.exemptFromIpFilter \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --stage=GA
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Bind this role to your user:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud projects add-iam-policy-binding $PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --member=&amp;#34;user:$GCPUSER&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --role=&amp;#34;projects/$PROJECT/roles/gcs_ipfilter_exempt&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Apply the Bucket IP filter. If you receive an error, wait a bit for the IAM update to propagate and try again:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud alpha storage buckets update gs://$BUCKET --ip-filter-file=allow-client-and-vpc-ip.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Updating gs://ipfilter-20241202/...
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  Completed 1  
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Using Cloud Shell, get the command to retrieve the file&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from terminal: &amp;#34; curl -s https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run that command on &lt;code&gt;node-allow&lt;/code&gt; and it will succeed, while on &lt;code&gt;node-deny&lt;/code&gt; and Cloud Shell it will continue to fail.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 data-hextra-search-id=&#34;remove-bucket-ip-filters&#34;&gt;Remove Bucket IP Filters&lt;/h1&gt;&lt;p&gt;Bucket IP filters can be disabled and cleared to revert to the default bucket configuration.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Using Cloud Shell, disable and clear Bucket IP filters:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud alpha storage buckets update gs://$BUCKET --clear-ip-filter
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Show the commands to check reachability:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo -e &amp;#34;\n&amp;#34; 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from terminal: &amp;#34; curl -s https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;Use from browser:  &amp;#34; https://tools.keycdn.com/performance?url=https://storage.googleapis.com/$BUCKET/hello.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Cut-n-paste the first output above and verify access from Cloud Shell. Next, click the second from the output above and observe the file can be opened from your web browser. Lastly, click the third from the output above, then the blue Test button, and you should see the file is accessed from a variety of locations on the Internet with a 200 OK HTTP Status code.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;conclusion&#34;&gt;Conclusion&lt;/h1&gt;&lt;p&gt;Cloud Storage offers bucket IP filtering to limit access to buckets based on the source IP address of the request. It enables additional control and the ability to limit by IP range for both Internet and VPC network clients for more control.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Building a shared GFS2 filesystem with Hyperdisk Balanced HA multi-writer</title>
      <link>https://www.beginswithdata.com/2024/10/08/hyperdisk-shared-gfs2/</link>
      <pubDate>Tue, 08 Oct 2024 08:00:00 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2024/10/08/hyperdisk-shared-gfs2/</guid>
      <description>
        
        
        &lt;p&gt;Building highly available and performant applications often requires shared storage accessible by multiple VMs. Traditionally shared storage has been made available to VMs in a single zone and was not fault tolerant to zone failure. Google Cloud&amp;rsquo;s &lt;a href=&#34;https://cloud.google.com/compute/docs/disks/sharing-disks-between-vms#hd-multi-writer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hyperdisk Balanced HA multi-writer&lt;/a&gt; technology solves this problem by combining cross-zone replication with multi-writer access from VMs across two zones.&lt;/p&gt;
&lt;p&gt;This solution opens up new possibilities for applications requiring both resilience and performance including:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Host based clusters:&lt;/strong&gt; High availability designs that typically use Linux Pacemaker or Windows Server Failover Clusters to assign and move resources amongst nodes in the cluster. With Hyperdisk Balanced HA multi-writer those nodes can now be in two different zones and use &lt;a href=&#34;https://cloud.google.com/compute/docs/disks/sharing-disks-between-vms#io-fencing&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Persistent Reservations&lt;/a&gt; to control access to shared-disk resources.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Shared-disk filesystems:&lt;/strong&gt; Shared storage designs supported by filesystems such as GFS2, OCFS2 and VMFS that manage access to shared-disk resources using clustering and/or locking capabilities for data coherence across nodes. Often combined with host based clusters.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If your shared storage needs can be met using NFS or SMB protocols I highly recommend using &lt;a href=&#34;https://cloud.google.com/filestore/docs/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Filestore&lt;/a&gt; (single zone or HA across three zones) or &lt;a href=&#34;https://cloud.google.com/netapp/volumes/docs/discover/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Volumes&lt;/a&gt; (single zone or HA across two zones). Both of these provide check-box simplicity for shared storage requirements in a zone, or across zones. But, the blog today is about Hyperdisk Multi-writer so we&amp;rsquo;ll take the harder path and build something ourselves!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;building-a-gfs2-shared-filesystem-with-ubuntu&#34;&gt;Building a GFS2 Shared Filesystem with Ubuntu&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;building-a-gfs2-shared-filesystem-with-ubuntu&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#building-a-gfs2-shared-filesystem-with-ubuntu&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Solutions that use shared disks tend to be complex, and to focus on the core concepts I decided to build a basic &lt;a href=&#34;https://en.wikipedia.org/wiki/GFS2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;GFS2&lt;/a&gt; shared-disk filesystem across two VMs in different zones. With this solution an application running on each of those VMs can read and write data from the same filesystem concurrently, and a VM (or zone) can fail and the data is still available in the other. Filesystem coherency is managed by a cluster-wide locking scheme that uses &lt;a href=&#34;https://en.wikipedia.org/wiki/Distributed_lock_manager&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DLM&lt;/a&gt; and &lt;a href=&#34;https://en.wikipedia.org/wiki/Corosync_Cluster_Engine&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Corosync&lt;/a&gt; to exchange locking information. I added a VM in a third zone to make locking services resilient.&lt;/p&gt;
&lt;p&gt;The infrastructure can be visualized as:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-10-08/design.png&#34; title=&#34;Multi-zone GFS2 design on GCP&#34; alt=&#34;Multi-zone GFS2 design on GCP&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Multi-zone GFS2 design on GCP&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;step-by-step-instructions&#34;&gt;Step by Step Instructions&lt;/h1&gt;&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-amber-200 hx:bg-amber-100 hx:text-amber-900 hx:dark:border-amber-200/30 hx:dark:bg-amber-900/30 hx:dark:text-amber-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; xmlns=&#34;http://www.w3.org/2000/svg&#34; fill=&#34;none&#34; viewBox=&#34;0 0 24 24&#34; stroke-width=&#34;2&#34; stroke=&#34;currentColor&#34; aria-hidden=&#34;true&#34;&gt;&lt;path stroke-linecap=&#34;round&#34; stroke-linejoin=&#34;round&#34; d=&#34;M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z&#34;/&gt;&lt;/svg&gt;&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;&lt;strong&gt;WARNING:&lt;/strong&gt; These instructions are useful to prove the concept and NOT suitable for a production deployment. A production deployment would secure Corosync, likely add Pacemaker cluster software for coordinated resource management and fencing to increase availability and fault tolerance, use LVM, and include tuning of the DLM and GFS2 settings for the desired use case.&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;h2 data-hextra-search-id=&#34;prepare-the-environment-and-create-cloud-resources&#34;&gt;Prepare the environment and create cloud resources&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;prepare-the-environment-and-create-cloud-resources&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#prepare-the-environment-and-create-cloud-resources&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Assumptions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;You are a project owner&lt;/li&gt;
&lt;li&gt;Avoid setting public IP addresses on resources, instead:
&lt;ul&gt;
&lt;li&gt;Configure a &lt;a href=&#34;https://cloud.google.com/nat/docs/set-up-manage-network-address-translation#set_up_a_simple_configuration&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Public NAT&lt;/a&gt; in the region enabling VMs to pull packages from the Internet&lt;/li&gt;
&lt;li&gt;Enable IAP for TCP forwarding and created a &lt;a href=&#34;https://cloud.google.com/iap/docs/using-tcp-forwarding#create-firewall-rule&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;firewall rule allowing IAP connections&lt;/a&gt; to your VPC&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Set environment variables for our desired location:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PROJECT=p20241007-gfs2-88
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;REGION=europe-west4
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The Dynamic Lock Manager (DLM) service and Corosync manage locking information between VMs in the cluster. Create a firewall rule to allow all communication between VMs when tagged with &lt;code&gt;gfs2cluster&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute firewall-rules create gfs2cluster \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--direction=INGRESS --allow=all \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--target-tags=gfs2cluster \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--source-tags=gfs2cluster
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create three Ubuntu VMs, one in each zone of your selected region. These VMs will form the basis of the GFS2 cluster:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;for ID in a b c; do
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    gcloud compute instances create node-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --machine-type=c3-highcpu-8 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --shielded-secure-boot \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --network-interface=stack-type=IPV4_ONLY,subnet=default,no-address \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --tags=gfs2cluster \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --zone=$REGION-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --create-disk=auto-delete=yes,boot=yes,mode=rw,provisioned-iops=3000,provisioned-throughput=140,size=10,image-project=ubuntu-os-cloud,image-family=ubuntu-2404-lts-amd64,type=projects/$PROJECT/zones/$REGION-$ID/diskTypes/hyperdisk-balanced
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;done
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a HdB-HA-MW disk with replicas in the &lt;code&gt;b&lt;/code&gt; and &lt;code&gt;c&lt;/code&gt; zones. We will use these zones for our GFS2 VMs and the VM in zone &lt;code&gt;a&lt;/code&gt; for distributed locking only.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute disks create hdb-ha-mw \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--type=hyperdisk-balanced-high-availability \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--replica-zones=projects/$PROJECT/zones/$REGION-b,projects/$PROJECT/zones/$REGION-c \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--size=250GB \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--access-mode=READ_WRITE_MANY \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--provisioned-iops=6000 --provisioned-throughput=280
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Attach the disk to the &lt;code&gt;-b&lt;/code&gt; and &lt;code&gt;-c&lt;/code&gt; VMs.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk node-b --disk=hdb-ha-mw --device-name=hdb-ha-mw --zone=$REGION-b --disk-scope=regional --project=$PROJECT
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk node-c --disk=hdb-ha-mw --device-name=hdb-ha-mw --zone=$REGION-c --disk-scope=regional --project=$PROJECT 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;install-and-configure-the-vms&#34;&gt;Install and configure the VMs&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;install-and-configure-the-vms&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#install-and-configure-the-vms&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Open a shell to each of the three VMs. Launching a SSH-in-browser tab from Cloud Console is easy.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On &lt;code&gt;node-a&lt;/code&gt; install modules needed for distributed locking and cluster communications:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo apt-get update
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo apt-get install corosync dlm-controld -y
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On &lt;code&gt;node-b&lt;/code&gt; and &lt;code&gt;node-c&lt;/code&gt; install modules for GFS2, distributed locking and cluster communications:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo apt-get update
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo apt-get install corosync dlm-controld gfs2-utils linux-modules-extra-$(uname -r) fio -y
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a Corosync config file based on your VMs. Update the below replacing the &lt;code&gt;node&lt;/code&gt; entries in the &lt;code&gt;nodelist&lt;/code&gt; with the FQDN and ip address of your VMs. Use &lt;code&gt;hostname&lt;/code&gt; and &lt;code&gt;ip addr show&lt;/code&gt; on each VM to get the needed values. Once you have adapted the config run the command to apply it on all three VMs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cat &amp;gt; corosync.conf &amp;lt;&amp;lt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;totem {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        version: 2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        cluster_name: gcpcluster
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        crypto_cipher: none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        crypto_hash: none
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;nodelist {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        node {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                name: node-a.europe-west4-a.c.p20241007-gfs2-88.internal
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                nodeid: 1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                ring0_addr: 10.164.0.7
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        }
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        node {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                name: node-b.europe-west4-b.c.p20241007-gfs2-88.internal
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                nodeid: 2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                ring0_addr: 10.164.0.8
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        }
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        node {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                name: node-c.europe-west4-c.c.p20241007-gfs2-88.internal
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                nodeid: 3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                ring0_addr: 10.164.0.9
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        }
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;quorum {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        provider: corosync_votequorum
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;system {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        allow_knet_handle_fallback: yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;logging {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        fileline: off
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        to_stderr: yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        to_logfile: yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        logfile: /var/log/corosync/corosync.log
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        to_syslog: yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        debug: off
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        logger_subsys {
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                subsys: QUORUM
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;                debug: off
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;        }
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;}
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo cp corosync.conf /etc/corosync/corosync.conf
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On each VM restart &lt;code&gt;corosync&lt;/code&gt; and verify all three nodes are members. Do not continue if the output does not show all three nodes as members:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo systemctl restart corosync
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo corosync-quorumtool 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Quorum information
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;------------------
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Date:             Fri Oct  4 16:48:01 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Quorum provider:  corosync_votequorum
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Nodes:            3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Node ID:          1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Ring ID:          1.12
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Quorate:          Yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Votequorum information
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;----------------------
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Expected votes:   3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Highest expected: 3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Total votes:      3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Quorum:           2  
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Flags:            Quorate 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Membership information
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;----------------------
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    Nodeid      Votes Name
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;         1          1 node-a.europe-west4-a.c.p20241007-gfs2-88.internal (local)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;         2          1 node-b.europe-west4-b.c.p20241007-gfs2-88.internal
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;         3          1 node-c.europe-west4-c.c.p20241007-gfs2-88.internal
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On &lt;code&gt;node-b&lt;/code&gt; create a GFS2 volume. The &lt;code&gt;-t&lt;/code&gt; argument includes the &lt;code&gt;cluster_name&lt;/code&gt; we configured in the &lt;code&gt;corosync.conf&lt;/code&gt; file, and &lt;code&gt;shared&lt;/code&gt; is a unique filesystem name.  We set &lt;code&gt;-j 2&lt;/code&gt; because we are sharing the disk with two VMs. Run this command and answer yes when prompted:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo mkfs.gfs2 -p lock_dlm -t gcpcluster:shared -j 2 /dev/disk/by-id/google-hdb-ha-mw
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On both &lt;code&gt;node-b&lt;/code&gt; and &lt;code&gt;node-c&lt;/code&gt; create a mount point and systemd service to mount it on boot because a standard entry in &lt;code&gt;/etc/fstab&lt;/code&gt; is insufficient due to dependencies on DLM.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo mkdir /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cat &amp;gt; gfs2-mount.service &amp;lt;&amp;lt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;[Unit]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Description=Mount GFS2 Filesystem
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;After=network-online.target
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;After=dlm.service
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;[Service]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Type=oneshot
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;RemainAfterExit=yes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;ExecStart=/bin/mount -t gfs2 -o rw,noatime,nodiratime,rgrplvb /dev/disk/by-id/google-hdb-ha-mw /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;ExecStop=/bin/umount /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;[Install]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;WantedBy=multi-user.target
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;EOF
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo cp gfs2-mount.service /etc/systemd/system/gfs2-mount.service
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo systemctl enable --now gfs2-mount.service
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;To ensure all required services start correctly on boot, stop and start all VMs from the Cloud Console or from the CLI using:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;for ID in a b c; do
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    gcloud compute instances stop node-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --zone=$REGION-$ID
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;done
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;for ID in a b c; do
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    gcloud compute instances start node-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --project=$PROJECT \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --zone=$REGION-$ID \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --async
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;done
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;mount-and-use-the-shared-filesystem&#34;&gt;Mount and use the shared filesystem&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;mount-and-use-the-shared-filesystem&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#mount-and-use-the-shared-filesystem&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Re-open shells to each of the three VMs. From &lt;code&gt;node-b&lt;/code&gt; relax filesystem permissions to ease access and then start a simple workload:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;sudo chmod 777 /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cd /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;watch -n 1 &amp;#34;echo -n $(hostname) &amp;#39; &amp;#39; &amp;gt;&amp;gt; date.out; date &amp;gt;&amp;gt; date.out&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;On &lt;code&gt;node-c&lt;/code&gt; read the file that is being appended to by &lt;code&gt;node-b&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;cd /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;tail -f date.out
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:20:49 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:20:50 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:20:51 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:20:52 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:20:53 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Let&amp;rsquo;s test this shared filesystem by running the same workload, on the same file, from &lt;code&gt;node-c&lt;/code&gt;. This will demonstrate how GFS2&amp;rsquo;s locking mechanisms ensure data integrity:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;watch -n 1 &amp;#34;echo -n $(hostname) &amp;#39; &amp;#39; &amp;gt;&amp;gt; date.out; date &amp;gt;&amp;gt; date.out&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Open another shell to &lt;code&gt;node-b&lt;/code&gt; and follow the file to observe writes from both hosts are intermixed:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;tail -f /shared/date.out
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-c.europe-west4-c.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:49 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:49 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-c.europe-west4-c.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:50 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:50 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-c.europe-west4-c.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:51 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b.europe-west4-b.c.p20241007-gfs2-88.internal  Mon Oct  7 16:22:51 UTC 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Now it&amp;rsquo;s time to test the performance. Stop the &lt;code&gt;date&lt;/code&gt; commands and use the &lt;code&gt;fio&lt;/code&gt; command below to start a write IOP workload from each VM concurrently. Observe each VM achieves half the disk provisioned IOPs, or 3K IOPs in our case, with similar latencies:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;fio --name=`hostname -s` --size=1G \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--time_based --runtime=5m --ramp_time=2s --ioengine=libaio --direct=1 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--verify=0 --bs=4K --iodepth=2 --rw=randwrite --group_reporting=1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;fio-3.36
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Starting 1 process
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Jobs: 1 (f=1): [w(1)][100.0%][w=11.7MiB/s][w=3001 IOPS][eta 00m:00s]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;node-b: (groupid=0, jobs=1): err= 0: pid=2501: Mon Oct  7 16:42:21 2024
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  write: IOPS=3016, BW=11.8MiB/s (12.4MB/s)(3535MiB/300001msec); 0 zone resets
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    slat (nsec): min=1919, max=150331, avg=4637.11, stdev=2043.55
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    clat (usec): min=404, max=29166, avg=657.96, stdev=192.52
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     lat (usec): min=407, max=29171, avg=662.60, stdev=192.85
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    clat percentiles (usec):
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     |  1.00th=[  474],  5.00th=[  506], 10.00th=[  529], 20.00th=[  553],
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     | 30.00th=[  570], 40.00th=[  586], 50.00th=[  611], 60.00th=[  627],
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     | 70.00th=[  652], 80.00th=[  685], 90.00th=[  955], 95.00th=[ 1123],
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     | 99.00th=[ 1270], 99.50th=[ 1319], 99.90th=[ 1483], 99.95th=[ 1713],
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     | 99.99th=[ 2671]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;   bw (  KiB/s): min=11704, max=14024, per=100.00%, avg=12068.29, stdev=300.79, samples=600
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;   iops        : min= 2926, max= 3506, avg=3017.06, stdev=75.19, samples=600
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  lat (usec)   : 500=4.09%, 750=82.75%, 1000=3.81%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  lat (msec)   : 2=9.32%, 4=0.03%, 10=0.01%, 20=0.01%, 50=0.01%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  cpu          : usr=0.32%, sys=1.57%, ctx=596311, majf=0, minf=37
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  IO depths    : 1=0.0%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &amp;gt;=64=0.0%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;gt;=64=0.0%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &amp;gt;=64=0.0%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     issued rwts: total=0,904999,0,0 short=0,0,0,0 dropped=0,0,0,0
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;     latency   : target=0, window=0, percentile=100.00%, depth=2
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Run status group 0 (all jobs):
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  WRITE: bw=11.8MiB/s (12.4MB/s), 11.8MiB/s-11.8MiB/s (12.4MB/s-12.4MB/s), io=3535MiB (3707MB), run=300001-300001msec
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Disk stats (read/write):
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  nvme0n2: ios=0/911244, sectors=0/7289952, merge=0/0, ticks=0/594190, in_queue=594190, util=98.94%
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Use Cloud Monitoring to verify the per attachment IOPs &lt;code&gt;compute.googleapis.com/instance/disk/write_ops_count&lt;/code&gt; metric, filtered on &lt;code&gt;device_name=hdb-ha-mw&lt;/code&gt;, aggregated by system metadata label &lt;code&gt;name&lt;/code&gt;:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-10-08/metrics.png&#34; title=&#34;Cloud Monitoring IOPs metrics&#34; alt=&#34;Cloud Monitoring IOPs metrics&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Monitoring IOPs metrics&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;At this point you are free to try other workloads, reboot to simulate failure modes, and in general experiment with the ins and outs of a shared-disk filesystem. When you are done, delete the VMs and the shared disk to clean up.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;conclusion&#34;&gt;Conclusion&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;conclusion&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#conclusion&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;The ability to replicate data across zones, coupled with the multi-writer access mode, makes Hyperdisk an ideal solution for applications demanding both high availability and performance. This blog post demonstrated a basic GFS2 implementation, but the possibilities extend far beyond. Hyperdisk Balanced HA multi-writer provides a foundation for building resilient and scalable applications on Google Cloud.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>High throughput file transfers with Google Cloud Storage (GCS)</title>
      <link>https://www.beginswithdata.com/2024/02/01/google-cloud-storage-max-throughput/</link>
      <pubDate>Thu, 01 Feb 2024 09:00:00 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2024/02/01/google-cloud-storage-max-throughput/</guid>
      <description>
        
        
        &lt;blockquote&gt;
  &lt;p&gt;&lt;strong&gt;TLDR;  Accelerate large single file GCS transfers from 200 MiB/s for a single stream, to over 8 GiB/s with parallel streams&lt;/strong&gt;&lt;/p&gt;

&lt;/blockquote&gt;
&lt;p&gt;This guide will teach you how to parallelize your Google Cloud Storage (GCS) file transfers for dramatically increased throughput vs. single stream transfers. We&amp;rsquo;ll cover GCS API choices, optimized upload/download strategies, library and tool support, and best practices. Let&amp;rsquo;s get to it!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;gcs-api-support&#34;&gt;GCS API Support&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;gcs-api-support&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#gcs-api-support&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Google Cloud Storage (GCS) offers a &lt;a href=&#34;https://cloud.google.com/storage/docs/json_api&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JSON API&lt;/a&gt; (Google&amp;rsquo;s own specification) and an &lt;a href=&#34;https://cloud.google.com/storage/docs/xml-api/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;XML API&lt;/a&gt; (compatible with AWS S3). Both are robust and suitable for demanding workloads, but there are subtle differences in their feature set. &lt;strong&gt;&lt;em&gt;Usually&lt;/em&gt;&lt;/strong&gt; the API used can be considered an implementation detail of your client libraries. If you’re using third party libraries or a utility with S3 support, just set the GCS service endpoint and potentially adapt some headers (&lt;code&gt;x-amz-storage-class&lt;/code&gt; ⇨ &lt;code&gt;x-goog-storage-class&lt;/code&gt;) and you&amp;rsquo;re good to go. Or if you&amp;rsquo;re developing something new with the &lt;a href=&#34;https://cloud.google.com/storage/docs/reference/libraries&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud Storage client libraries&lt;/a&gt;, the &lt;a href=&#34;https://firebase.google.com/docs/storage/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Firebase libraries&lt;/a&gt;, or using the &lt;a href=&#34;https://cloud.google.com/sdk/gcloud/reference/storage&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;gcloud storage&lt;/a&gt; CLI, &lt;strong&gt;&lt;em&gt;usually&lt;/em&gt;&lt;/strong&gt; the JSON API is leveraged.&lt;/p&gt;
&lt;p&gt;I said &lt;strong&gt;&lt;em&gt;usually&lt;/em&gt;&lt;/strong&gt; twice above, and why I did that will become apparent in short order. 😀&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;fast-uploads&#34;&gt;Fast Uploads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;fast-uploads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#fast-uploads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Achieving fast uploads for large files requires these three steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Logically split your file into byte-range chunks&lt;/li&gt;
&lt;li&gt;Upload the chunks in parallel&lt;/li&gt;
&lt;li&gt;Assemble the chunks to create the large file&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Visually it looks like this:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/parallel-upload-flow.png&#34; title=&#34;GCS Object Upload&#34; alt=&#34;GCS Object Upload&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;GCS Object Upload&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;To achieve this parallelization there are two techniques: &lt;a href=&#34;https://cloud.google.com/storage/docs/parallel-composite-uploads&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Composite Uploads&lt;/a&gt; (JSON and XML API) and &lt;a href=&#34;https://cloud.google.com/storage/docs/multipart-uploads&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Multipart Uploads&lt;/a&gt; (XML API). While the basic concept of splitting and assembling is the same, the implementation differs. As a result, each technique has different limits and interoperability with other GCS features. Depending on your use case, both might do the job equally well, or one might be a better fit.&lt;/p&gt;
&lt;p&gt;Lets walk through the flow of the Composite Upload pattern available with the JSON and XML API:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/composite-upload-flow.png&#34; title=&#34;GCS Composite Upload&#34; alt=&#34;GCS Composite Upload&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;GCS Composite Upload&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;First, the file is logically split into byte ranges producing a number of chunks. Next, the chunks are uploaded in parallel as normal files to a temporary filename that is visible in the bucket. After all chunk files have been uploaded, a compose operation is issued that creates a new file from these chunks. Finally, the temporary chunk files can be deleted.&lt;/p&gt;
&lt;p&gt;Now let&amp;rsquo;s walk through the flow of the Multipart Upload pattern available only with the XML API:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/multipart-upload-flow.png&#34; title=&#34;GCS Multipart Upload&#34; alt=&#34;GCS Multipart Upload&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;GCS Multipart Upload&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;First, the file is logically split into byte ranges producing a number of chunks. Next, a Multipart Upload is initiated for the desired filename, and an &lt;code&gt;uploadId&lt;/code&gt; is returned. Then, the chunks are uploaded in parallel as a &lt;code&gt;partNumber&lt;/code&gt; of the &lt;code&gt;uploadId&lt;/code&gt; into a hidden area of the bucket. After all chunks have been uploaded, a finalize operation creates a new file from the &lt;code&gt;partNumber&lt;/code&gt; chunks, and also cleans up by deleting all chunk files associated with the &lt;code&gt;uploadId&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Here are the differences when it comes to implementation and interoperability:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Composite Uploads (JSON, XML)&lt;/strong&gt;&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Multipart Upload (XML)&lt;/strong&gt;&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Treatment of chunk files&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Normal files in the bucket&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Special part files associated with an upload and stored in a hidden area of the bucket&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Maximum number of chunks&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;32, but the resulting file can itself be a chunk for a subsequent Compose operation iteratively without limit&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;10,000&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Minimum chunk size&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;None&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;5 MB for all chunks except the last one&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Operations cost&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Class A operations = (# chunks uploaded + 1 compose)&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Class A operations = (# chunks uploaded + 1 create + 1 finalize).&lt;br&gt;If you choose a very small chunk size the ops costs can add up!&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;IAM permissions required&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;storage.objects.create&lt;/code&gt;&lt;br&gt;&lt;code&gt;storage.objects.delete&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;storage.objects.create&lt;/code&gt;&lt;br&gt;&lt;code&gt;storage.multipartUploads.create&lt;/code&gt;&lt;br&gt;&lt;code&gt;storage.multipartUploads.abort&lt;/code&gt;&lt;br&gt;&lt;code&gt;storage.multipartUploads.listParts&lt;/code&gt;&lt;br&gt;&lt;code&gt;storage.multipartUploads.list&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Works well with bucket retention, holds, and soft delete&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No. Applies to chunk and final files&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes. Applies to final file only&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Works well with colder storage classes&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No. Minimum storage duration (and any early delete fees) is applicable for chunk files and final file. As an workaround, upload at STANDARD and update the storage class of the final file thereafter&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes. Minimum storage duration (and any early delete fees) is applicable for final file only allowing upload at the desired colder storage class directly&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Change notifications&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Triggered for chunk files and final file&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Triggered only for final file&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Resumable upload support&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No, but you can set the chunk size as low as 5 MiB to limit the amount of data that might need to be retransmitted&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Automatic clean up of temporary chunks&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No. Chunks must be deleted by client logic and/or by using an Object Lifecycle Manager (OLM) delete lifecycle action that removes older files that match a well-known prefix&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes. A successful finalize or abort request deletes all chunks. Additionally, an Object Lifecycle Manager (OLM) &lt;code&gt;AbortIncompleteMultipartUpload&lt;/code&gt; lifecycle action can be used to abort stale requests and delete their chunks&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Likelihood to hit rate limits with technique&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;An object can be updated at most once per second. Ensure client logic, especially during repetitive composes, does not update the same object more frequently&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No rate limits specific to Multipart Uploads&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;Google Cloud Storage client library support&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;All programming languages&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;transfer_manager&lt;/code&gt; module in the &lt;a href=&#34;https://cloud.google.com/storage/docs/multipart-uploads#storage-upload-object-chunks-nodejs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Node.js&lt;/a&gt; and &lt;a href=&#34;https://cloud.google.com/storage/docs/multipart-uploads#storage-upload-object-chunks-python&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Python&lt;/a&gt; libraries&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;strong&gt;gcloud storage utility support&lt;/strong&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Yes&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;No&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;In summary, the Composite Upload pattern combines several distinct APIs to enable parallel uploads, whereas the Multipart Upload API is purpose built for it. As a result, Composite Uploads generally require more advanced client logic and don&amp;rsquo;t interoperate with GCS features including retention, holds, soft delete, and colder storage classes.&lt;/p&gt;
&lt;p&gt;If Multipart Uploads are better in pretty much every way, why would you use Composite Uploads?  The answer is simple: &lt;strong&gt;Client library and utility support&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;But no fear, because Multipart Uploads are part of the XML API you can use client libraries or utilities that are S3 compatible, or use the &lt;code&gt;transfer_manager&lt;/code&gt; module in the &lt;a href=&#34;https://cloud.google.com/storage/docs/multipart-uploads#storage-upload-object-chunks-nodejs&#34;&gt;Node.js&lt;/a&gt; and &lt;a href=&#34;https://cloud.google.com/storage/docs/multipart-uploads#storage-upload-object-chunks-python&#34;&gt;Python&lt;/a&gt; Google Cloud Storage libraries. Calling the XML API directly for these uploads is also an option.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;fast-downloads&#34;&gt;Fast Downloads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;fast-downloads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#fast-downloads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Achieving fast downloads for large files requires these three steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Logically split your file into byte-range chunks&lt;/li&gt;
&lt;li&gt;Download the chunks in parallel&lt;/li&gt;
&lt;li&gt;Assemble the chunks to create the large file&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Visually it looks like this:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/parallel-download-flow.png&#34; title=&#34;GCS Parallel Download&#34; alt=&#34;GCS Parallel Download&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;GCS Parallel Download&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;Because your target is typically a file on a POSIX compatible filesystem, you can parallelize writes of different chunks directly to the relevant offsets in your local file. From an API perspective you are simply issuing range read GET requests against the large file on GCS. Both the &lt;a href=&#34;https://cloud.google.com/storage/docs/xml-api/reference-headers#range&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JSON&lt;/a&gt; and &lt;a href=&#34;https://cloud.google.com/storage/docs/json_api/v1/parameters#range&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;XML&lt;/a&gt; APIs support range reads and can be used with comparable performance and feature interoperability.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;common-gcs-transfer-tools--libraries&#34;&gt;Common GCS transfer tools / libraries&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;common-gcs-transfer-tools--libraries&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#common-gcs-transfer-tools--libraries&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;While there are many tools out there, the &lt;code&gt;gcloud storage&lt;/code&gt; CLI and the &lt;a href=&#34;https://cloud.google.com/storage/docs/reference/libraries&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud Storage libraries&lt;/a&gt; remain the most common way to interact with GCS. Each has its own behaviors that you need to understand when designing for parallelism and peak performance. As a side note, I also tried s5cmd, rclone, and boto3, but was unable to match the performance possible with &lt;code&gt;gcloud storage&lt;/code&gt; or the GCS Python libraries. Your mileage may vary.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;gcloud-storage&#34;&gt;gcloud storage&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;gcloud-storage&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#gcloud-storage&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;&lt;code&gt;gcloud storage&lt;/code&gt; can parallelize uploads and downloads using multiple workers and calls the JSON API under the covers. For uploads it uses Composite Uploads when possible, and for downloads it uses range reads. It automatically parallelizes transfers by calculating an optimal chunk size and number of workers to transfer those chunks. You can also set a number of variables for uploads (&lt;code&gt;parallel_composite_upload_*&lt;/code&gt;), downloads ( &lt;code&gt;sliced_object_download_*&lt;/code&gt;), and workers (&lt;code&gt;process_count&lt;/code&gt; and &lt;code&gt;thread_count&lt;/code&gt;) as described in the &lt;a href=&#34;https://cloud.google.com/sdk/gcloud/reference/config&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;gcloud config set&lt;/code&gt;&lt;/a&gt; documentation. I tried tweaking these settings to try and squeeze even more performance out but couldn&amp;rsquo;t improve on the default behavior.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;google-cloud-storage-python-libraries&#34;&gt;Google Cloud Storage Python Libraries&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;google-cloud-storage-python-libraries&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#google-cloud-storage-python-libraries&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;The &lt;a href=&#34;https://cloud.google.com/storage/docs/reference/libraries&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud Storage libraries&lt;/a&gt; make it easier to work with GCS and are available for C++, C#, Go, Java, Node.js, PHP, Python, and Ruby. All of these can be used to upload files in parallel using the Compose pattern and your own multiprocessing logic. If you&amp;rsquo;re using Python or Node.js, these libraries also include a &lt;code&gt;transfer_manager&lt;/code&gt; module that uses the XML Multipart API for uploads, and the JSON API for downloads, and handles the multiprocessing logic for you. Set the &lt;code&gt;chunk_size&lt;/code&gt; and &lt;code&gt;max_workers&lt;/code&gt; parameters and the module takes care of the rest.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;testing-time&#34;&gt;Testing Time&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;testing-time&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#testing-time&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; fill=&#34;currentColor&#34; viewBox=&#34;3 3 18 18&#34;&gt;
  &lt;path d=&#34;M12 3C7.0275 3 3 7.12937 3 12.2276C3 16.3109 5.57625 19.7597 9.15374 20.9824C9.60374 21.0631 9.77249 20.7863 9.77249 20.5441C9.77249 20.3249 9.76125 19.5982 9.76125 18.8254C7.5 19.2522 6.915 18.2602 6.735 17.7412C6.63375 17.4759 6.19499 16.6569 5.8125 16.4378C5.4975 16.2647 5.0475 15.838 5.80124 15.8264C6.51 15.8149 7.01625 16.4954 7.18499 16.7723C7.99499 18.1679 9.28875 17.7758 9.80625 17.5335C9.885 16.9337 10.1212 16.53 10.38 16.2993C8.3775 16.0687 6.285 15.2728 6.285 11.7432C6.285 10.7397 6.63375 9.9092 7.20749 9.26326C7.1175 9.03257 6.8025 8.08674 7.2975 6.81794C7.2975 6.81794 8.05125 6.57571 9.77249 7.76377C10.4925 7.55615 11.2575 7.45234 12.0225 7.45234C12.7875 7.45234 13.5525 7.55615 14.2725 7.76377C15.9937 6.56418 16.7475 6.81794 16.7475 6.81794C17.2424 8.08674 16.9275 9.03257 16.8375 9.26326C17.4113 9.9092 17.76 10.7281 17.76 11.7432C17.76 15.2843 15.6563 16.0687 13.6537 16.2993C13.98 16.5877 14.2613 17.1414 14.2613 18.0065C14.2613 19.2407 14.25 20.2326 14.25 20.5441C14.25 20.7863 14.4188 21.0746 14.8688 20.9824C16.6554 20.364 18.2079 19.1866 19.3078 17.6162C20.4077 16.0457 20.9995 14.1611 21 12.2276C21 7.12937 16.9725 3 12 3Z&#34;&gt;&lt;/path&gt;
&lt;/svg&gt;
&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;The benchmarking script used to test the &lt;code&gt;gcloud storage&lt;/code&gt; CLI and the &lt;code&gt;transfer_manager&lt;/code&gt; module from the Google Cloud Storage Python client library is in this GitHub repository:&lt;br&gt;
&lt;strong&gt;&lt;a href=&#34;https://github.com/dutchiechris/storage/tree/main/gcs/gcs-bench-single&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;github.com/dutchiechris/storage/tree/main/gcs/gcs-bench-single&lt;/a&gt;&lt;/strong&gt;&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;Okay, enough theory, time for practice! Using the benchmarking script above, I measured the throughput of the different transfer techniques using both the &lt;code&gt;gcloud storage&lt;/code&gt; CLI and the &lt;code&gt;transfer_manager&lt;/code&gt; module from the Google Cloud Storage Python client library. For these tests I used a &lt;strong&gt;c3-highmem-8&lt;/strong&gt; and a &lt;strong&gt;c3-standard-176&lt;/strong&gt; machine and copied data to and from a ramdisk to avoid disk bottlenecks and put the focus on the APIs and libraries. Especially at the higher throughput levels the disks (PD, Hyperdisk, Local SSD) will need to be sized appropriately. Lastly, some performance variation is normal. All results reported are the mean of 10 test runs where I discarded the best and worst values.&lt;/p&gt;
&lt;h4 data-hextra-search-id=&#34;serial-uploads&#34;&gt;Serial Uploads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;serial-uploads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#serial-uploads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;For these tests I used a 3 GiB file. To force gcloud to operate serially I set &lt;code&gt;CLOUDSDK_STORAGE_PARALLEL_COMPOSITE_UPLOAD_ENABLED&lt;/code&gt; to &lt;code&gt;False&lt;/code&gt;. Note: if your bucket is configured with retention, object holds, or uses a storage class other than STANDARD,  &lt;code&gt;gcloud storage&lt;/code&gt; will set this to &lt;code&gt;False&lt;/code&gt; automatically to avoid interoperability issues with the chunk files. For Python I used the &lt;code&gt;upload_from_filename&lt;/code&gt; method.&lt;/p&gt;
&lt;p&gt;Here are the results:&lt;/p&gt;
&lt;table style=&#34;border: 3px dashed grey; border-collapse: separate; border-spacing: 10px;&#34;&gt;
  &lt;tr&gt;
   &lt;td style=&#34;width:70%&#34;&gt; &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/serial-upload.png&#34; alt=&#34;Serial Upload Throughput&#34;&gt;&lt;/td&gt;
   &lt;td&gt;&lt;strong&gt;Observations:&lt;/strong&gt;&lt;br&gt;Appears to be typical single stream throughput delivered by GCS. Adding more vCPU, or using a different copy utility has no impact.&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;h4 data-hextra-search-id=&#34;serial-downloads&#34;&gt;Serial Downloads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;serial-downloads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#serial-downloads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;For these tests I used a 3 GiB file. To force &lt;code&gt;gcloud&lt;/code&gt; to operate serially I set &lt;code&gt;CLOUDSDK_STORAGE_SLICED_OBJECT_DOWNLOAD_THRESHOLD&lt;/code&gt; to &lt;code&gt;0&lt;/code&gt;. For Python I used the &lt;code&gt;download_to_filename&lt;/code&gt; method.&lt;/p&gt;
&lt;p&gt;Here are the results:&lt;/p&gt;
&lt;table style=&#34;border: 3px dashed grey; border-collapse: separate; border-spacing: 10px;&#34;&gt;
  &lt;tr&gt;
   &lt;td style=&#34;width:70%&#34;&gt; &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/serial-download.png&#34; alt=&#34;Serial Download Throughput&#34;&gt;&lt;/td&gt;
   &lt;td&gt;&lt;strong&gt;Observations:&lt;/strong&gt;&lt;br&gt;Appears to be typical single stream throughput delivered by GCS. Adding more vCPU, or using a different copy utility has no impact.&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;h4 data-hextra-search-id=&#34;parallel-uploads&#34;&gt;Parallel Uploads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;parallel-uploads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#parallel-uploads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;For these tests I used either a 40 GiB (8 vCPU VM) or 80 GiB (176 vCPU VM) file. For &lt;code&gt;gcloud&lt;/code&gt; I allowed it to optimize the parallelization settings, and for Python I used the &lt;code&gt;upload_chunks_concurrently&lt;/code&gt; method with a 25 MiB chunk size and either 50 (c3-highmem-8) or 176 (c3-standard-176) workers.&lt;/p&gt;
&lt;p&gt;Here are the results:&lt;/p&gt;
&lt;table style=&#34;border: 3px dashed grey; border-collapse: separate; border-spacing: 10px;&#34;&gt;
  &lt;tr&gt;
   &lt;td style=&#34;width:70%&#34;&gt; &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/parallel-upload.png&#34; alt=&#34;Parallel Upload Throughput&#34;&gt;&lt;/td&gt;
   &lt;td&gt;&lt;strong&gt;Observations:&lt;/strong&gt;&lt;br&gt;Here we see the benefit of increased parallelism possible with the Multipart XML API found in the &lt;code&gt;transfer_manager&lt;/code&gt; module of the GCS Python libraries. On the 8 vCPU VM the difference is marginal because the 32 parallel streams possible with the Compose pattern uses most of the VM resources, but on the larger shape we can drive much higher throughput with greater parallelism.&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;h4 data-hextra-search-id=&#34;parallel-downloads&#34;&gt;Parallel Downloads&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;parallel-downloads&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#parallel-downloads&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;For these tests I used either a 40 GiB (8 vCPU VM) or 80 GiB (176 vCPU VM) file. When using &lt;code&gt;gcloud storage&lt;/code&gt; I allowed it to optimize the parallelization settings, and for Python I used the &lt;code&gt;download_chunks_concurrently&lt;/code&gt; method with a 25 MiB chunk size and either 50 (c3-highmem-8) or 176 (c3-standard-176) workers.&lt;/p&gt;
&lt;p&gt;Here are the results:&lt;/p&gt;
&lt;table style=&#34;border: 3px dashed grey; border-collapse: separate; border-spacing: 10px;&#34;&gt;
  &lt;tr&gt;
   &lt;td style=&#34;width:70%&#34;&gt; &lt;img src=&#34;https://www.beginswithdata.com/post/2024-02-01/parallel-download.png&#34; alt=&#34;Parallel Download Throughput&#34;&gt;&lt;/td&gt;
   &lt;td&gt;&lt;strong&gt;Observations:&lt;/strong&gt;&lt;br&gt;&lt;code&gt;gcloud storage&lt;/code&gt; consistently delivered greater throughput than the &lt;code&gt;transfer_manager&lt;/code&gt; module when saving to a file on the ramdisk. However when saving to &lt;code&gt;/dev/null&lt;/code&gt; to approximate a direct load to application memory, we can achieve a much higher throughput level. I suspect the throughput when saving to a file is gated by file or filesystem concurrency inherent in the Python implementation found in these two tools.&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;h3 data-hextra-search-id=&#34;recommendations&#34;&gt;Recommendations&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;recommendations&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#recommendations&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;GCS is highly performant and delivers over 200 MiB/s streaming per transfer request for uploads or downloads. Greater throughput requires parallelization and the VM vCPU count, network limit, and storage limits will become your gating factors. For uploads, the Multipart Upload XML API delivers the highest throughput with the lowest implementation effort, and provides the best compatibility with GCS features. For downloads, the XML and JSON APIs with range reads are basically equivalent.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Read-only persistent disks on Google Cloud</title>
      <link>https://www.beginswithdata.com/2023/11/20/google-cloud-read-only-disk/</link>
      <pubDate>Mon, 20 Nov 2023 08:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2023/11/20/google-cloud-read-only-disk/</guid>
      <description>
        
        
        &lt;blockquote&gt;
  &lt;p&gt;&lt;strong&gt;TLDR;  High performance low cost sharing of static data with read-only persistent disk attachments&lt;/strong&gt;&lt;/p&gt;

&lt;/blockquote&gt;
&lt;p&gt;Both Google Cloud Compute Engine and Kubernetes Engine allow you to attach a Persistent Disk (PD) in either read-write or read-only mode. For the Compute Engine VM boot disk, you’ll usually create it from an existing image or snapshot and attach as read-write. Data disks on the other hand can be created empty, or seeded with data from an existing image or snapshot, and are usually also attached in read-write mode.&lt;/p&gt;
&lt;p&gt;But what if your instance or container on GKE needs &lt;strong&gt;read-only access&lt;/strong&gt; to [potentially Terabytes of] reference data? This is when a read-only PD can come in quite handy!&lt;/p&gt;
&lt;p&gt;First though, you should seriously evalute solutions designed for sharing: &lt;a href=&#34;https://cloud.google.com/storage/docs/introduction&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud Storage&lt;/a&gt;, perhaps using &lt;a href=&#34;https://cloud.google.com/storage/docs/gcs-fuse&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;GCSfuse&lt;/a&gt;, &lt;a href=&#34;https://cloud.google.com/filestore/docs/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Filestore&lt;/a&gt; with NFS, or &lt;a href=&#34;https://cloud.google.com/netapp/volumes/docs/discover/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud NetApp Volumes&lt;/a&gt; with NFS and/or SMB. All of these can present data read-only to some clients, while allowing updates from others, and are generally easier to use and maintain.&lt;/p&gt;
&lt;p&gt;If those solutions don&amp;rsquo;t work for you, perhaps because you need local filesystem semantics, you can get &lt;strong&gt;high-performance access without incremental cost&lt;/strong&gt; using the read-only attachment mode like this:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-11-20/attachments.png&#34; title=&#34;Persistent disk attachment modes&#34; alt=&#34;Persistent disk attachment modes&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Persistent disk attachment modes&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;A few things to be aware of:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Attachment modes are mutually exclusive: a disk is attached to up to one instance in read-write mode OR attached to one or more instances in read-only mode&lt;/li&gt;
&lt;li&gt;Standard HDD, Balanced SSD, and Performance SSD disk types support read-only attachments&lt;/li&gt;
&lt;li&gt;Google recommends &lt;strong&gt;no more than 10 read-only attachments per PD&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;You will be charged for the capacity of the PD only; the read-only attachments are not charged&lt;/li&gt;
&lt;li&gt;Read-only ext3 and ext4 filesystems do not support journal playback to make the filesystem consistent. While the &lt;code&gt;noload&lt;/code&gt; mount option can be used to skip loading of the journal, there may be filesystem inconsistencies that can lead to any number of problems. Unmounting the filesystem before creating a clone or snapshot avoids these issues.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;While you can prepare a disk attached read-write on a single instance, detach, and attach as read-only on multiple instances, it&amp;rsquo;s usually easier to &lt;a href=&#34;https://cloud.google.com/compute/docs/disks/create-disk-from-source&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;create a new one&lt;/a&gt; from a clone or [instant] snapshot of your original read-write attached disk.&lt;/p&gt;
&lt;p&gt;There are some nuances depending on the Operating System, and if you&amp;rsquo;re running as VMs or on GKE. Let&amp;rsquo;s walk through three scenarios:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#linux-vms-and-read-only-pd-attachments&#34;&gt;Linux VMs and read-only PD attachments&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#google-kubernetes-engine-gke-and-read-only-pd-attachments&#34;&gt;Google Kubernetes Engine (GKE) and read-only PD attachments&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#windows-server-vms-and-read-only-pd-attachments&#34;&gt;Windows Server VMs and read-only PD attachments&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h1 data-hextra-search-id=&#34;linux-vms-and-read-only-pd-attachments&#34;&gt;Linux VMs and read-only PD attachments&lt;/h1&gt;&lt;p&gt;Linux allows usage of read-only disks and filesystems via mount options. The steps are quite straightforward.&lt;/p&gt;
&lt;p&gt;Assuming you already have a Linux VM running, create a new disk to hold the data you want to share and attach it to your VM. I really like the &lt;code&gt;--device-name&lt;/code&gt; flag of the &lt;code&gt;attach-disk&lt;/code&gt; subcommand as it gives us a predictable device file to use later:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute disks create shared --type=pd-balanced --size=1024GB --zone=europe-west4-b 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk base-vm --disk=shared --device-name=shared --zone=europe-west4-b 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Login the VM, format the disk, copy your data to it, and then unmount it to ensure a clean filesystem is captured:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;mkdir /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;mkfs.ext3 /dev/disk/by-id/google-shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;mount /dev/disk/by-id/google-shared /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;echo &amp;#34;shared data goes here&amp;#34; &amp;gt; /shared/data-to-share.txt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;unmount /shared
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Next create a new disk from it that will be shared to multiple VMs with a &lt;code&gt;ro&lt;/code&gt; attachment&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute disks create shared-ro --source-disk=shared --zone=europe-west4-b
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk worker1 --disk=shared-ro --device-name=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk worker2 --disk=shared-ro --device-name=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk worker3 --disk=shared-ro --device-name=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;And then on each worker VM mount it readonly:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;mkdir /shared-ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;mount -o ro /dev/disk/by-id/google-shared-ro /shared-ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;That&amp;rsquo;s it, you now have read-only access to the dataset on all your worker VMs!&lt;/p&gt;
&lt;p&gt;If you want to be able to write to the disk, such as to run a pre-populated database for many training or test VMs, perhaps it could be combined with a writable layer using a union filesystem. This investigation is left as an exercise for the reader 😉.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;google-kubernetes-engine-gke-and-read-only-pd-attachments&#34;&gt;Google Kubernetes Engine (GKE) and read-only PD attachments&lt;/h1&gt;&lt;p&gt;Google Kubernetes Engine (GKE) includes the Compute Engine persistent disk CSI Driver to provision and manage storage. Storage can be requested using a &lt;code&gt;PersistentVolumeClaim&lt;/code&gt; with &lt;code&gt;accessModes&lt;/code&gt; &lt;code&gt;ReadWriteOnce&lt;/code&gt; for a writable volume that can be mounted by a single node, and &lt;code&gt;ReadOnlyMany&lt;/code&gt; for a read-only volume that can be mounted by multiple nodes. Similar to the other scenarios, it&amp;rsquo;s easiest to create a new disk to hold your source data as writable, write data to it, and then clone it to another disk for read only attachments. These steps can all be done directly by GKE. Let&amp;rsquo;s get started!&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Create a GKE cluster&lt;/a&gt; and then &lt;a href=&#34;https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;install and configure kubectl&lt;/a&gt; access.&lt;/p&gt;
&lt;p&gt;If you already have a disk with the data you want to share on it, check these &lt;a href=&#34;https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/preexisting-pd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;instructions&lt;/a&gt; on how to bring it into Kubernetes. If not,  prepare a &lt;code&gt;writer&lt;/code&gt; container with some storage attached. Do this by creating a file named &lt;code&gt;rw.yaml&lt;/code&gt; containing:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;Namespace&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;apiVersion&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;labels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nn&#34;&gt;---&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;PersistentVolumeClaim&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;apiVersion&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;namespace&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;shared&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;accessModes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;- &lt;span class=&#34;l&#34;&gt;ReadWriteOnce&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;storageClassName&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;standard-rwo&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;resources&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;requests&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;storage&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;1Ti&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nn&#34;&gt;---&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;apiVersion&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;apps/v1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;Deployment&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;writer&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;namespace&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;labels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;writer&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;replicas&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;selector&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;matchLabels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;writer&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;template&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;labels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;writer&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;containers&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;writer&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;image&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx:latest&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;volumeMounts&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;mountPath&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;/usr/share/nginx/html&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;mypvc&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;ports&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;containerPort&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;m&#34;&gt;80&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;volumes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;mypvc&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;persistentVolumeClaim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;claimName&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;shared&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;readOnly&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;false&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;This yaml contains three resource definitions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;Namespace&lt;/code&gt;: It&amp;rsquo;s a best practice to put resourecs in their own namespace, so lets create one called &lt;code&gt;blog-ns&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;PersistentVolumeClaim&lt;/code&gt;: This is a request for a 1 TiB of balanced PD disk. The &lt;code&gt;accessModes&lt;/code&gt; of &lt;code&gt;ReadWriteOnce&lt;/code&gt; makes it writable and able to be mounted by a single node&lt;/li&gt;
&lt;li&gt;&lt;code&gt;Deployment&lt;/code&gt;: This is a spec for a simple container running an application that mounts the PersistentVolumeClaim&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Apply the configuration using:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl apply -f rw.yaml
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Use &lt;code&gt; kubectl get pods --namespace=blog-ns -o wide&lt;/code&gt; to check the details of the running pod. Normally you would have a real application that saves data to the volume that you want to share, but for demo purposes we&amp;rsquo;ll just save a text file to it and verify we can read it via the web server. Use the following and replace the &lt;code&gt;writer-647fc7b778-cjgtx&lt;/code&gt; with the name of your pod:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl exec writer-647fc7b778-cjgtx --namespace=blog-ns -- sh -c &amp;#39;echo &amp;#34;shared data goes here&amp;#34; &amp;gt; /usr/share/nginx/html/data-to-share.txt&amp;#39;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl exec writer-647fc7b778-cjgtx --namespace=blog-ns -- sh -c &amp;#39;curl -s http://localhost/data-to-share.txt&amp;#39;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;shared data goes here
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Delete the &lt;code&gt;writer&lt;/code&gt; deployment to ensure the disk and filesystem are unmounted and in a clean state:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl delete deployment writer --namespace=blog-ns
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Next, prepare some reader containers with a new cloned volume attached as &lt;code&gt;ReadOnlyMany&lt;/code&gt;. Do this by creating a file named &lt;code&gt;ro.yaml&lt;/code&gt; containing:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;PersistentVolumeClaim&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;apiVersion&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;namespace&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;shared-ro&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;dataSource&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;shared&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;PersistentVolumeClaim&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;accessModes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;- &lt;span class=&#34;l&#34;&gt;ReadOnlyMany&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;storageClassName&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;standard-rwo&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;resources&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;requests&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;storage&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;1Ti&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nn&#34;&gt;---&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;apiVersion&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;apps/v1&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;kind&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;Deployment&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;readers&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;namespace&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;blog-ns&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;labels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;replicas&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;m&#34;&gt;11&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;selector&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;matchLabels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;template&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;metadata&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;labels&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;app&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;spec&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;containers&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;image&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;nginx:latest&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;volumeMounts&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;mountPath&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;/usr/share/nginx/html&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;mypvc&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;ports&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;containerPort&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;m&#34;&gt;80&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;volumes&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;mypvc&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;persistentVolumeClaim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;claimName&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;shared-ro&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;readOnly&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;kc&#34;&gt;true&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;This yaml contains two resource definitions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;PersistentVolumeClaim&lt;/code&gt;: This is a request for a 1 TiB of balanced PD disk. The &lt;code&gt;dataSource&lt;/code&gt; parameter references the volume we created in the &lt;code&gt;rw.yaml&lt;/code&gt; spec and that disk will be cloned to create this new disk. The &lt;code&gt;accessModes&lt;/code&gt; of &lt;code&gt;ReadOnlyMany&lt;/code&gt; makes it readable and able to be mounted by multiple nodes&lt;/li&gt;
&lt;li&gt;&lt;code&gt;Deployment&lt;/code&gt;: This is a spec for a simple container running an application that mounts the PersistentVolumeClaim&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Apply the configuration using:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl apply -f ro.yaml
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Use &lt;code&gt; kubectl get pods --namespace=blog-ns -o wide&lt;/code&gt; to check the details of the running pods. All of these are accessing a read-only attachment of the cloned volume. Lets read the data we wrote earlier using the writer pod. Use the following and replace the &lt;code&gt;readers-5dcb555946-4w988&lt;/code&gt; with the name of your pod:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;kubectl exec readers-5dcb555946-4w988  --namespace=blog-ns -- sh -c &amp;#39;curl -s http://localhost/data-to-share.txt&amp;#39;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;shared data goes here
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Each node gets the full performance of the PD, allowing each of them to go fast!  Keep in mind that balanced PD can be read-only attached to at most 10 nodes. If GKE tries to attach the volume to an 11th node you will see pods stuck in the &lt;code&gt;ContainerCreating&lt;/code&gt; state with an error like this:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;AttachVolume.Attach failed for volume &amp;#34;pvc-c86f1b46-88ed-4e9f-b3f4-b9eb8380a2c0&amp;#34; : rpc error: code = Internal desc = Failed to Attach: failed cloud service attach disk call: 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;googleapi: Error 400: Invalid resource usage: &amp;#39;Attachment limit exceeded. Read-only persistent disks can be attached to at most 10 instances.&amp;#39;., invalidResourceUsage
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;If you run at a larger scale you can use multiple cloned disks, each of which can be attached to up to 10 nodes.&lt;/p&gt;
&lt;p&gt;You can clean up by deleting the namespace using &lt;code&gt;kubectl delete namespace blog-ns&lt;/code&gt;.&lt;/p&gt;
&lt;h1 data-hextra-search-id=&#34;windows-server-vms-and-read-only-pd-attachments&#34;&gt;Windows Server VMs and read-only PD attachments&lt;/h1&gt;&lt;p&gt;Windows Server supports read-only PD attachments but only if its on disk metadata has the &lt;code&gt;Read-only&lt;/code&gt; attribute on the disk or volume set to &lt;code&gt;True&lt;/code&gt;. This means that before you attach it as &lt;code&gt;ro&lt;/code&gt; you need to set the Disk or Volume attribute to &lt;code&gt;Read-only&lt;/code&gt; from a Windows Server perspective.&lt;/p&gt;
&lt;p&gt;Lets assume you have a Windows VM running, with a PD named &lt;code&gt;shared&lt;/code&gt; using a &lt;code&gt;rw&lt;/code&gt; attachment. Open a command prompt &lt;strong&gt;as Administrator&lt;/strong&gt; and set the data disk to be &lt;code&gt;readonly&lt;/code&gt; using &lt;code&gt;diskpart&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;C:\&amp;gt;diskpart
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Microsoft DiskPart version 10.0.20348.1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Copyright (C) Microsoft Corporation.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;On computer: WIN-MAIN
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;DISKPART&amp;gt; list disk
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  Disk ###  Status         Size     Free     Dyn  Gpt
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --------  -------------  -------  -------  ---  ---
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  Disk 0    Online           50 GB      0 B        *
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  Disk 1    Online         1024 GB  1024 KB        *
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;DISKPART&amp;gt; select disk 1
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Disk 1 is now the selected disk.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;DISKPART&amp;gt; attributes disk set readonly
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Disk attributes set successfully.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Now you can clone the disk and attach it to other hosts with a &lt;code&gt;ro&lt;/code&gt; mode attachment:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute disks create shared-ro --source-disk=shared --zone=europe-west4-b
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk win-student1 --disk=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk win-student2 --disk=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances attach-disk win-student3 --disk=shared-ro --zone=europe-west4-b --mode=ro
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;If all went well, you should see your drive and large dataset mounted read-only on all three VMs!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;bonus-write-access-on-windows-server-using-a-differencing-layer-on-top-of-the-read-only-pd&#34;&gt;Bonus: Write access on Windows Server using a differencing layer on top of the read-only PD&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;bonus-write-access-on-windows-server-using-a-differencing-layer-on-top-of-the-read-only-pd&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#bonus-write-access-on-windows-server-using-a-differencing-layer-on-top-of-the-read-only-pd&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;A customer asked me if they could somehow use a read-only disk as the base for a writeable disk, something like a VMware linked clone, or Linux qcow disk, for use in a training environment. They had several large databases and wanted to avoid the cost for many full copies of it during the training sessions. While I don&amp;rsquo;t think such a feature exists in Windows Disk Management, we can use Microsoft&amp;rsquo;s Virtual Hard Disk (VHD) to do it. In this scenario you create a fixed VHD on your PD, load the data you want to share into the VHD, and then detach the VHD. Next set the Windows disk to be &lt;code&gt;Read-only&lt;/code&gt; using &lt;code&gt;diskpart&lt;/code&gt; and clone the PD to another PD and attach as &lt;code&gt;ro&lt;/code&gt; to the student VMs. From the student VM, create a new differencing VHD on the boot drive with the parent VHD located on the &lt;code&gt;Read-only&lt;/code&gt; drive. Voila, from a user perspective you now have a writable database disk but avoided the cost of many large dedicated PDs! From a performance perspective I didn&amp;rsquo;t see any significant penalty for adding in the VHD layer; Reads were based on the disk limits of the &lt;code&gt;Read-only&lt;/code&gt; disk and writes based on the boot disk. Obviously this technique is not designed for performance or a long-running use case, but for some transient needs it could be useful.&lt;/p&gt;
&lt;p&gt;Here are the steps to do it. First, create your parent VHD on the read-write PD using these &lt;code&gt;diskpart&lt;/code&gt; commands:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;create vdisk file=&amp;#34;D:\base1.vhdx&amp;#34; type=fixed maximum=1000
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;select vdisk  file=&amp;#34;D:\base1.vhdx&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;attach vdisk
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;create partition primary
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;active
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;assign letter=S
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;format quick fs=ntfs label=database
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Now load the data you want to share onto the &lt;code&gt;S:&lt;/code&gt; drive and detach that VHD.  Then proceed with the steps explained earlier (i.e. set &lt;code&gt;Read-only&lt;/code&gt; from Windows Server, clone the PD to a new one, and map it out to the student VMs).&lt;/p&gt;
&lt;p&gt;Then, on the student VMs, create a new differencing VHD that is based on the parent VHD using these &lt;code&gt;diskpart&lt;/code&gt; commands:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;create vdisk file=&amp;#34;C:\local.vhdx&amp;#34; parent=&amp;#34;D:\base1.vhdx&amp;#34; 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;select vdisk file=&amp;#34;C:\local.vhdx&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;attach vdisk
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;You should see a new drive letter appear, and if you check the contents you&amp;rsquo;ll find a writeable drive with your database files in it:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-11-20/vhd.png&#34; title=&#34;Differencing VHD&#34; alt=&#34;Differencing VHD&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Differencing VHD&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;For 10 students who need their own writable 1 TiB database this technique just avoided 9 TiB of disk costs!&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Step-by-step tutorial: SMB file storage using Google Cloud NetApp Volumes</title>
      <link>https://www.beginswithdata.com/2023/09/10/google-cloud-netapp-volumes-smb/</link>
      <pubDate>Sun, 10 Sep 2023 08:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2023/09/10/google-cloud-netapp-volumes-smb/</guid>
      <description>
        
        
        &lt;p&gt;I just kicked the tires of &lt;a href=&#34;https://cloud.google.com/netapp/volumes/docs/discover/overview&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud NetApp Volumes service&lt;/a&gt; and it is pretty slick! This service is delivered by Google Cloud using NetApp ONTAP and has full integration with IAM, Cloud Monitoring, Cloud Logging, Cloud Console, and the Google Cloud SDK. If you need Windows file shares using SMB on Google Cloud then this is the managed service for you!&lt;/p&gt;
&lt;p&gt;There is a &lt;a href=&#34;https://cloud.google.com/netapp/volumes/docs/get-started/quickstarts/create-storage-pool&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;quickstart guide&lt;/a&gt; in the official documentation, but plenty of steps are &amp;ldquo;left to the reader&amp;rdquo;. While it&amp;rsquo;s pretty standard cloud configuration, the devil 😈 is in the details.&lt;/p&gt;
&lt;p&gt;To help folks try out this service I&amp;rsquo;ve documented the steps I took to configure my environment from scratch through to accessing a SMB file share from a Windows VM. All the steps you need including configuring networking and firewall rules, setting up dns, installing and configuring a Windows Active Directory Domain Controller, creating the NetApp Volumes resources, and finally testing from a Windows VM. I&amp;rsquo;ve also avoided the use of public IP addresses to reduce potential security lapses. While the steps here might not match your production deployment, they will at least show you how to build a working solution.&lt;/p&gt;
&lt;p&gt;Here&amp;rsquo;s what we&amp;rsquo;re going to build:
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/architecture.png&#34; title=&#34;NetApp Volumes Architecture&#34; alt=&#34;NetApp Volumes Architecture&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;NetApp Volumes Architecture&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Expected time to execute steps: 1 hour&lt;/em&gt;&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;initialize-the-project-and-set-defaults&#34;&gt;Initialize the project and set defaults&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;initialize-the-project-and-set-defaults&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#initialize-the-project-and-set-defaults&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;ll create a fresh project, associate a billing account, and enable the needed APIs.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://cloud.google.com/sdk/docs/install-sdk&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Install and initialize the gcloud CLI&lt;/a&gt; on your local machine if needed.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a fresh project:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud projects create smb-20230829 --organization&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;85556667778&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Link a billing account:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud beta billing projects link smb-20230829 --billing-account&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;000111-AAABBB-222333
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set the default &lt;code&gt;gcloud&lt;/code&gt; project so we don&amp;rsquo;t need to supply it on each command:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud config &lt;span class=&#34;nb&#34;&gt;set&lt;/span&gt; project smb-20230829
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enable the APIs needed to use Google Cloud Volumes and integrate with Active Directory:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud services &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; compute.googleapis.com &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;dns.googleapis.com \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;iap.googleapis.com \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;netapp.googleapis.com oslogin.googleapis.com \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;servicenetworking.googleapis.com
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;create-windows-server-vms-for-active-directory-and-a-test-client&#34;&gt;Create Windows Server VMs for Active Directory and a test client&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;create-windows-server-vms-for-active-directory-and-a-test-client&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#create-windows-server-vms-for-active-directory-and-a-test-client&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;ll create a Windows VMs for our Domain Controller (DC). We&amp;rsquo;ll also reserve the IP of the DC to ensure it doesn&amp;rsquo;t change. This is also the time when you&amp;rsquo;ll want to decide your GCP region, which for simplicity should match the &lt;a href=&#34;https://cloud.google.com/netapp/volumes/docs/locations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;location&lt;/a&gt; where you intend to use NetApp Volumes. I&amp;rsquo;m based in Amsterdam so I chose &lt;code&gt;euorpe-west4 (The Netherlands)&lt;/code&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Create the Windows VM. I&amp;rsquo;ve called the DC &lt;code&gt;adc1&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute instances create adc1 &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --zone=europe-west4-c \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --machine-type=e2-standard-2 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --network-interface=stack-type=IPV4_ONLY,subnet=default,no-address \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --tags=msft-adc \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --create-disk=auto-delete=yes,boot=yes,mode=rw,size=50,image-project=windows-cloud,image-family=windows-2022,type=projects/smb-20230829/zones/europe-west4-c/diskTypes/pd-balanced
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Reserve the DC&amp;rsquo;s &lt;a href=&#34;https://cloud.google.com/compute/docs/ip-addresses/reserve-static-internal-ip-address#promote-in-use-internal-address&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;in-use IP address&lt;/a&gt; so it won&amp;rsquo;t change.&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute instances describe adc1 --zone europe-west4-c &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; grep -E &lt;span class=&#34;s2&#34;&gt;&amp;#34;networkIP|subnetwork&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  networkIP: 10.164.0.3
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  subnetwork: https://www.googleapis.com/compute/v1/projects/smb-20230829/regions/europe-west4/subnetworks/default
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute addresses create adc1 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --addresses 10.164.0.3 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --region europe-west4 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --subnet default
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;configure-network-connection-between-your-vpc-and-the-netapp-volumes-service&#34;&gt;Configure network connection between your VPC and the NetApp Volumes Service&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;configure-network-connection-between-your-vpc-and-the-netapp-volumes-service&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#configure-network-connection-between-your-vpc-and-the-netapp-volumes-service&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;ll create a connection between your VPC and the NetApp Volumes service using private service access.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Reserve a static internal IP address range for use by the NetApp Volumes service. I chose &lt;code&gt;192.168.0.0/24&lt;/code&gt; for simplicity:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute addresses create netapp-addresses-default &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --global \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --purpose=VPC_PEERING \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --prefix-length=24 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --network=default \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --no-user-output-enabled \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;    --addresses 192.168.0.0
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Peer your VPC to the NetApp Volumes service using the network range you just created. The command will take a few minutes to complete:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud services vpc-peerings connect &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --service=netapp.servicenetworking.goog \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --ranges=netapp-addresses-default \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --network=default
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The previous step will result in the network peering &lt;code&gt;sn-netapp-prod&lt;/code&gt; being created for you. Next, enable custom route propagation on it:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute networks peerings update sn-netapp-prod &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --network=default  \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --import-custom-routes \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --export-custom-routes
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;add-firewall-rules&#34;&gt;Add firewall rules&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;add-firewall-rules&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#add-firewall-rules&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;ll add firewall rules needed for our overall scenario.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Add a firewall rule allowing the NetApp Volumes Service to communicate with your Microsoft Active Directory Domain Controller:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute firewall-rules create allow-netapp-peer-auth &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--description=&amp;#34;Allow Microsoft authentication traffic from NetApp peered network&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--direction=INGRESS --priority=1000 --network=default --action=ALLOW \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--rules=tcp:53,tcp:88,tcp:389,tcp:445,tcp:464,tcp:636,udp:53,udp:88,udp:464 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--source-ranges=192.168.0.0/24 --target-tags=msft-adc
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add a firewall rule allowing Google IAP connections into the VPC:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute firewall-rules create allow-google-iap &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--description=&amp;#34;Allow RDP traffic from Google IAP addresses&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--direction=INGRESS --priority=1000 --network=default --action=ALLOW \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--rules=tcp:3389 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--source-ranges=35.235.240.0/20
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add a firewall rule allowing Cloud DNS to forward requests to your Microsoft Active Directory Domain Controller:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute firewall-rules create allow-cloud-dns-forwarding &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--description=&amp;#34;Allow Cloud DNS request forwarding&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--direction=INGRESS --priority=1000 --network=default --action=ALLOW \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--rules=tcp:53,udp:53 \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;--source-ranges=35.199.192.0/19 --target-tags=msft-adc
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;install-and-configure-the-microsoft-active-directory-domain-controller&#34;&gt;Install and configure the Microsoft Active Directory Domain Controller&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;install-and-configure-the-microsoft-active-directory-domain-controller&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#install-and-configure-the-microsoft-active-directory-domain-controller&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;ll install a fresh DC using Powershell, but before we can do that we need to connect with IAP and reset Windows passwords in a few places.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Request Compute Engine to reset the Windows password of our own account to give credentials that will allow us to connect with RDP:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute reset-windows-password adc1 --zone europe-west4-c
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;This command creates an account and sets an initial password for the
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;user [chris] if the account does not already exist.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;If the account already exists, resetting the password can cause the
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;LOSS OF ENCRYPTED DATA secured with the current password, including
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;files and stored passwords.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;For more information, see:
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;https://cloud.google.com/compute/docs/operating-systems/windows#reset
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Would you like to set or reset the password for [chris] (Y/n)?  Y
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Resetting and retrieving password for [chris] on [adc1]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Updated [https://www.googleapis.com/compute/v1/projects/smb-20230829/zones/europe-west4-c/instances/adc1].
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;WARNING: Instance [adc1] does not appear to have an external IP
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;address, so it will not be able to accept external connections.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;To add an external IP address to the instance, use
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;gcloud compute instances add-access-config.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;password: h$+notrealJdid877d
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;username: chris
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Use &lt;a href=&#34;https://cloud.google.com/iap/docs/using-tcp-forwarding#tunneling_rdp_connections&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Identity-Aware Proxy (IAP) to tunnel RDP traffic&lt;/a&gt; from your local computer port 8000 to your Active Directory Controller port 3389:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud compute start-iap-tunnel adc1 &lt;span class=&#34;m&#34;&gt;3389&lt;/span&gt; --local-host-port&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;localhost:8000 --zone&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;europe-west4-c
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Testing if tunnel connection works.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Listening on port [8000].
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Use your RDP client to connect to &lt;code&gt;localhost:8000&lt;/code&gt; with your username and the password generated in the reset step 1 above.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Open Powershell as Administrator (right click on Powershell and choose run as administrator)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enable the local administrator and set a password&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PS C:\Windows\system32&amp;gt; set-localuser -Name Administrator -Password (Read-Host -AsSecureString)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;&amp;lt;input password&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PS C:\Windows\system32&amp;gt; Enable-LocalUser Administrator
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Install Active Directory and DNS. Tip: Use a domain name that won&amp;rsquo;t resolve on the Internet to avoid confusion. For example, I chose &lt;code&gt;smb.local&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PS C:\Windows\system32&amp;gt; Install-WindowsFeature -name AD-Domain-Services -IncludeManagementTools           
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Success Restart Needed Exit Code      Feature Result
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;------- -------------- ---------      --------------
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;True    No             Success        {Active Directory Domain Services, Group P...
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;PS C:\Windows\system32&amp;gt; Install-ADDSForest -DomainName &amp;#34;smb.local&amp;#34; -InstallDNS
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;SafeModeAdministratorPassword: *********
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Confirm SafeModeAdministratorPassword: *********
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;The target server will be configured as a domain controller and restarted when this operation is complete.
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Do you want to continue with this operation?
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is &amp;#34;Y&amp;#34;): Y
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;Ignore Warnings. System will restart. 
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Reconenct your RDP client to &lt;code&gt;localhost:8000&lt;/code&gt; but this time using your AD Domain credentials (in my case &lt;code&gt;smb\administrator&lt;/code&gt;) to verify the Domain works properly.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Configure DNS request forwarding so that Cloud DNS will forward requests for AD managed zones to it. This steps will ensure that VMs are able to do lookups related to the Windows Domain while still using the default Cloud DNS resolver. Note the &lt;code&gt;forwarding-targets&lt;/code&gt; parameter is the IP address of the DC:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-console&#34; data-lang=&#34;console&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;gp&#34;&gt;$&lt;/span&gt; gcloud dns managed-zones create smb-local &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --description=&amp;#34;Forward requests for the Microsoft AD managed domain&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --dns-name=&amp;#34;smb.local.&amp;#34; --visibility=&amp;#34;private&amp;#34; \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --networks=default \
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;go&#34;&gt;  --forwarding-targets=&amp;#34;10.164.0.3&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;configure-netapp-volumes&#34;&gt;Configure NetApp Volumes&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;configure-netapp-volumes&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#configure-netapp-volumes&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this section we&amp;rsquo;re going to configure NetApp Volumes using the Cloud Console.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;First we need to configure the AD settings. In this step you define the configuration for your Active Directory environment. Note that it doesn&amp;rsquo;t actually establish a connection and having a valid policy doesn&amp;rsquo;t actually mean that it will actually work 🙈. In Cloud Console go to NetApp Volumes, and then &lt;strong&gt;Active Directory policies&lt;/strong&gt;, and then Create. The input fields are fairly intuitive and you can find the values from your earlier configuration steps:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/create-ad-policy.png&#34; title=&#34;Create AD Policy&#34; alt=&#34;Create AD Policy&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Create AD Policy&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Next we need to create a Storage pool. A Storage pool exists in a region, has a service level and capacity, is connected to a VPC, and is assigned an Active Directory policy. To configure a new Storage pool click &lt;strong&gt;Storage pool&lt;/strong&gt; and then Create.&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/create-pool.png&#34; title=&#34;Create Storage Pool&#34; alt=&#34;Create Storage Pool&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Create Storage Pool&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Finally, it&amp;rsquo;s time to create a volume on the Storage pool. A volume has a capacity that consumes space from the Storage pool, has a share name, and can be presented out on different file protocols. When creating a SMB share there are a few more options you can try out. Under snapshots I recommend you set a basic schedule and make the snapshot directory visible so your end-users can restore data easily. To create the volume go to the &lt;strong&gt;Volumes&lt;/strong&gt; tab and then Create:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/create-volume.png&#34; title=&#34;Create Volume&#34; alt=&#34;Create Volume&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Create Volume&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The first SMB volume may take a few minutes to create because in the background a connection from NetApp Volumes to your DC needs to be established, and a computer account for the file server created. Once it completes, check the mount instructions here:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/mount-instructions.png&#34; title=&#34;Mounted instructions&#34; alt=&#34;Mount instructions&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Mounted instructions&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;In your RDP session, mount the share! 🎉 🍻:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2023-09-10/mounted-drive.png&#34; title=&#34;Mounted Drive&#34; alt=&#34;Mounted Drive&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Mounted Drive&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;I hope this step-by-step guide helps you to understand the individual steps needed to use the NetApp Volumes service, from zero to hero!&lt;/p&gt;
&lt;p&gt;If you&amp;rsquo;d like to keep going, I suggest creating a Windows Server client VM and testing from it as well. To do this create another Windows Server VM, reset the user password, bring up an IAP tunnel to its RDP port, and connect to it using RDP. Then join that new VM to the AD Domain you created (which works because of the DNS forwarding rule!) and allow it to reboot. Thereafter you can re-establish an RDP session using your domain credientials and mount the share just like you did from the DC directly.&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Using Google BigQuery Sandbox: A tutorial</title>
      <link>https://www.beginswithdata.com/2019/11/09/2019-11-09-using-google-bigquery-sandbox-tutorial/</link>
      <pubDate>Sat, 09 Nov 2019 17:00:27 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2019/11/09/2019-11-09-using-google-bigquery-sandbox-tutorial/</guid>
      <description>
        
        
        &lt;p&gt;I recently presented a session about Google BigQuery at the Google Cloud Summit in Amsterdam.
While preparing for the session I was reminded how easy it is to get started, &lt;strong&gt;at no cost and without a credit card&lt;/strong&gt;, to explore your own data.
In my presentation I showed a short demo working with a dataset I uploaded, and then combining it with a public dataset. This blog allows you to execute the demo yourself, for free!&lt;/p&gt;
&lt;p&gt;In this 30 minute tutorial you will get experience with:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Accessing Google Cloud and the free BigQuery sandbox&lt;/li&gt;
&lt;li&gt;Loading your own data from Cloud Shell&lt;/li&gt;
&lt;li&gt;Querying and visualizing your own data&lt;/li&gt;
&lt;li&gt;Using the geospatial capabilities of BigQuery to explore public weather data&lt;/li&gt;
&lt;li&gt;Combining your own and public weather data to make new visualizations&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Are you ready?  Let&amp;rsquo;s go!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;1-login-to-google-cloud&#34;&gt;[1] Login to Google Cloud&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;1-login-to-google-cloud&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#1-login-to-google-cloud&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;First thing is to get connected to Google Cloud.  Use your existing Google (Gmail) account, or create a new one just to try this stuff out.  Remember, you will not be charged anything, nor have to provide payment details to use the &lt;a href=&#34;https://cloud.google.com/bigquery/docs/sandbox&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BigQuery Sandbox&lt;/a&gt; which gives you monthly up to 1 TiB of querying and 10 GiB of space for tables with a 60 day retention.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Load the &lt;a href=&#34;https://console.cloud.google.com/bigquery&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BigQuery UI&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you want to create a new account do so now, otherwise login using your existing account.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Accept the Google Cloud Platform Terms of Service.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Click on the &lt;strong&gt;create project&lt;/strong&gt; link and give it a name.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Done! You should see the BigQuery UI query editor ready to go!&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/1.png&#34; title=&#34;BigQuery UI Sandbox&#34; alt=&#34;BigQuery UI Sandbox&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery UI Sandbox&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;2-run-a-query-on-a-public-dataset&#34;&gt;[2] Run a query on a public dataset&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;2-run-a-query-on-a-public-dataset&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#2-run-a-query-on-a-public-dataset&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;There are tons of &lt;a href=&#34;https://cloud.google.com/bigquery/public-data/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;public datasets&lt;/a&gt; already loaded in BigQuery, so lets use one to make sure our environment is working.
My personal favorite is a dataset of birth names in the USA since 1910.  We can feed our curiousity and make sure things work at the same time :-)&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;In the BigQuery UI enter the following query, replacing &amp;ldquo;Chris&amp;rdquo; with [part] of your own name to find names similar to your own, and hit Run:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;gender&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;SUM&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;number&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;total&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquery&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;public&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;usa_names&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;usa_1910_current&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;where&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;like&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Chris%&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;GROUP&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;gender&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ORDER&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;total&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;DESC&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;LIMIT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;100&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;You should see results that look like something like this:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/2.png&#34; title=&#34;Baby names starting with Chris&#34; alt=&#34;Baby names starting with Chris&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Baby names starting with Chris&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Celebrate that you name is more special than you thought, or cry if it&amp;rsquo;s less!  Now it&amp;rsquo;s time to load some of your own data.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;3-load-some-data&#34;&gt;[3] Load some data.&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;3-load-some-data&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#3-load-some-data&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Since I&amp;rsquo;m in The Netherlands I started scouting around on some open data sites and stubled upon parking transactions for the city of &lt;a href=&#34;https://en.wikipedia.org/wiki/Arnhem&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Arnhem&lt;/a&gt;. Hmm.
I always wondered when parking garages are busiest so why not give it a try?&lt;/p&gt;
&lt;p&gt;There are several ways to load data into BigQuery, with the most common one being from Cloud Storage. But, Cloud Storage requires at least a Google Cloud trial account [and payment details], and we didn&amp;rsquo;t do that.  You can upload data from the BigQuery UI directly, but it&amp;rsquo;s limited to 10 MiB per file. Cloud Shell to the rescue!  Cloud Shell is a free linux shell you can launch from the Cloud Console that will enable us to load bigger files.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Open up a Cloud Shell using the shell icon from the Google Console&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/3a.png&#34; title=&#34;Cloud Shell Icon&#34; alt=&#34;Cloud Shell Icon&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Cloud Shell Icon&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;In a new browser tab, open up the &lt;a href=&#34;https://www.parkeerdata.nl/opendata/arnhem/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Arnhem parking data&lt;/a&gt;, and find the table that has &lt;strong&gt;Transactiedata - Parkeergarages&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Download a file to Cloud Shell by right clicking on a parking data file, copy link, and paste into a wget command in your Cloud Shell.  This is just one of the files, grab as many as you like:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget https://monitdata.s3.amazonaws.com/arnhem/offstreet/transactions/arnhem_opendata_offstreet_201908.csv&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a BigQuery table for the parking data.  Click on the project name and then the &lt;strong&gt;CREATE DATASET&lt;/strong&gt; button&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/3b.png&#34; title=&#34;Create dataset&#34; alt=&#34;Create dataset&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Create dataset&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set a dataset name of &lt;strong&gt;parking&lt;/strong&gt;, do not modify the other options, and click &lt;strong&gt;Create dataset&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;From Cloud Shell we can create a table and load the data we downloaded a moment ago:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;bq load &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;--source_format&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;CSV &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;--skip_leading_rows&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;1&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;--field_delimiter&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;;&amp;#39;&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;--replace&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;false&lt;/span&gt; &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;--autodetect &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;parking.data &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;./arnhem_opendata_offstreet_201908.csv&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;A few seconds later your data is ready to be queried in BigQuery!  Since we created the table from the CLI, refresh the browser tab for it to list the new table.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Expand the &lt;strong&gt;Resources&lt;/strong&gt; tree to show the table, and click on the &lt;strong&gt;Schema&lt;/strong&gt;, &lt;strong&gt;Details&lt;/strong&gt;, and &lt;strong&gt;Preview&lt;/strong&gt; tabs to get a better understanding of the data.&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/3c.png&#34; title=&#34;BigQuery table preview&#34; alt=&#34;BigQuery table preview&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery table preview&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;4-query-your-data&#34;&gt;[4] Query your data&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;4-query-your-data&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#4-query-your-data&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Clicking on the table, and then the preview tab, is a nice way to get a feel for the data format. I decided I&amp;rsquo;d like to see which day of the week is the busiest for each garage, for non subscription holders, and here&amp;rsquo;s the query I came up.  Run it after changing the &lt;strong&gt;bigqueryusersandbox&lt;/strong&gt; to your own project name:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;count&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;parking_visits&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_nm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;FORMAT_TIMESTAMP&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;%a&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;start_parking_dt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weekday&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquerysandboxuser&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;parking&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;WHERE&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;start_parking_dt&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NOT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;end_parking_dt&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NOT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;card_type_nm&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;Kort parkeren&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;GROUP&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;weekday&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_nm&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ORDER&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_nm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;parking_visits&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;DESC&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The results indicate Saturday, Thursday, and Friday are the busiest!&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/4.png&#34; title=&#34;BigQuery results showing garage visits per day of the week&#34; alt=&#34;BigQuery results showing garage visits per day of the week&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery results showing garage visits per day of the week&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;5-visualize-your-data&#34;&gt;[5] Visualize your data&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;5-visualize-your-data&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#5-visualize-your-data&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Scrolling through data in a table can be nice, but lets make it visual using Google Data Studio.  Data Studio is a free data visualization tool from Google that has great integration with BigQuery!
Let&amp;rsquo;s try to make a stacked bar chart with visits per day of the week per garage.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Click the &lt;strong&gt;Explore with Data Studio&lt;/strong&gt; button and click through the prompts that appear.&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/5a.png&#34; title=&#34;Explore with Data Studio&#34; alt=&#34;Explore with Data Studio&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Explore with Data Studio&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The Data Studio interface &lt;strong&gt;Data&lt;/strong&gt; tab is where you need to match up the graph view type and the source data from the table.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;First, click a stacked column chart&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/5b.png&#34; title=&#34;Stacked Column Chart&#34; alt=&#34;Stacked Column Chart&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Stacked Column Chart&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Next, from the &lt;strong&gt;available fields&lt;/strong&gt; list, drag and drop &lt;strong&gt;weekday&lt;/strong&gt; to the &lt;strong&gt;Dimension&lt;/strong&gt; field of your chart&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/5c.png&#34; title=&#34;Dimension is weekday&#34; alt=&#34;Dimension is weekday&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Dimension is weekday&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Next, from the &lt;strong&gt;available fields&lt;/strong&gt; list, drag and drop &lt;strong&gt;parking_visits&lt;/strong&gt; to the &lt;strong&gt;Metric&lt;/strong&gt; field of your chart&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/5d.png&#34; title=&#34;Metric is parking_visits&#34; alt=&#34;Metric is parking_visits&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Metric is parking_visits&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Check out the beautiful chart that visualizes the data.  Looks like for the data I loaded the parking garage users are coming for fun and not work!&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/5e.png&#34; title=&#34;Stacked bar chart of parking visits per day of the week per garage&#34; alt=&#34;Stacked bar chart of parking visits per day of the week per garage&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Stacked bar chart of parking visits per day of the week per garage&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;It&amp;rsquo;s possible to save this chart, and then share with other users, with a few more clicks. Easy does it!&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;6-combine-your-data-with-public-data&#34;&gt;[6] Combine your data with public data&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;6-combine-your-data-with-public-data&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#6-combine-your-data-with-public-data&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Now that we have our own parking data loaded, lets try to combine it with &lt;a href=&#34;https://cloud.google.com/blog/products/gcp/global-historical-daily-weather-data-now-available-in-bigquery&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Global Historical Climate Network (GHCN)&lt;/a&gt;  weather data, one of the &lt;a href=&#34;https://cloud.google.com/bigquery/public-data/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;public datasets&lt;/a&gt; already loaded in BigQuery.
This dataset includes historical weather data from over 80,000 stations in 180 countries.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;First we need to find a weather station nearby Arnhem that is reporting.  To do that, we need the latitude and longitude of the Centraal garage in Arnhem. An easy way is to &lt;a href=&#34;https://goo.gl/maps/WSbFGfAvSwkQ7w8a7&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;search&lt;/a&gt; for it in Google Maps, and then right click on the garage and choose the &amp;ldquo;What&amp;rsquo;s here&amp;rdquo;.  A small window appears with a photo of the location with the decimal latitude and longitude of it.&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6a.png&#34; title=&#34;How to find latitude and longitude of a location using Google Maps&#34; alt=&#34;How to find latitude and longitude of a location using Google Maps&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;How to find latitude and longitude of a location using Google Maps&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;In BigQuery search for the closest 20 weather stations based on this latitude and longitude:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;id&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;latitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;longitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ST_DISTANCE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ST_GEOGPOINT&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;longitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;latitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;),&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ST_GEOGPOINT&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;900425&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;51&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;984002&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1000&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;km_to_station&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ST_GEOGPOINT&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;longitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;latitude&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;as&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;WKT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquery&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;public&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcn_d&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcnd_stations&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ORDER&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;km_to_station&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;ASC&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;LIMIT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;20&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;You can also see this visually using &lt;a href=&#34;https://bigquerygeoviz.appspot.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BigQuery Geo Viz&lt;/a&gt;.  Visit this site, authorize it and pick your GCP project, paste in the same query, and set the Style -&amp;gt; circleRadius to 500:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6b.png&#34; title=&#34;BigQuery Geo Viz&#34; alt=&#34;BigQuery Geo Viz&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery Geo Viz&lt;/figcaption&gt;
  &lt;/figure&gt;
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6c.png&#34; title=&#34;BigQuery Geo Viz Results&#34; alt=&#34;BigQuery Geo Viz Results&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery Geo Viz Results&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Not all stations have current weather, so take a quick look to make sure the station name you pick (in our case NLE00152497) is reporting:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;MAX&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mm_precipitation&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mm_precipitation&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;MAX&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmin&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmin&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;MAX&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmax&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmax&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IF&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;element&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;PRCP&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;10&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;mm_precipitation&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IF&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;element&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;TMIN&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;10&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmin&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IF&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;element&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;TMAX&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;10&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;tmax&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquery&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;public&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcn_d&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcnd_2019&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;WHERE&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;id&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;NLE00152497&amp;#39;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;qflag&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;GROUP&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ORDER&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;&lt;em&gt;Note: Check here for a &lt;a href=&#34;https://towardsdatascience.com/using-bigquerys-new-geospatial-functions-to-interpolate-temperatures-d9363ff19446&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;query&lt;/a&gt; that helps find reporting stations automatically&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;And, for fun, click the &lt;strong&gt;EXPLORE WITH DATA STUDIO&lt;/strong&gt; button and setup the as below for a nice chart:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6d.png&#34; title=&#34;BigQuery Geo Viz&#34; alt=&#34;BigQuery Geo Viz&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery Geo Viz&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Now we can combine our parking data and the weather data:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;DATE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;start_parking_dt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;sdate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;count&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;*&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;parking_visits&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_nm&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prcp&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;SELECT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;value&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;10&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prcp&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;FROM&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquery&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;public&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcn_d&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;ghcnd_2019&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;WHERE&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;id&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;NLE00152497&amp;#39;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;qflag&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;element&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s1&#34;&gt;&amp;#39;PRCP&amp;#39;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;JOIN&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;bigquerysandboxuser&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;parking&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;k&#34;&gt;data&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;`&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;p&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ON&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;DATE&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;start_parking_dt&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;nb&#34;&gt;date&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;WHERE&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;start_parking_dt&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NOT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;AND&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;end_parking_dt&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;IS&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NOT&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;NULL&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;GROUP&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;garage_name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;sdate&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;prcp&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;ORDER&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;k&#34;&gt;BY&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;n&#34;&gt;sdate&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6e.png&#34; title=&#34;BigQuery results with parking and percipitation&#34; alt=&#34;BigQuery results with parking and percipitation&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;BigQuery results with parking and percipitation&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;And finally, click the &lt;strong&gt;EXPLORE WITH DATA STUDIO&lt;/strong&gt; button and lets see if there is any obvious relationship between it being wet and the amount of parking visits:&lt;/p&gt;
&lt;p&gt;&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6f.png&#34; title=&#34;Data Studio settings and visualization for parking visits and percipitation -1-&#34; alt=&#34;Data Studio settings and visualization for parking visits and percipitation -1-&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Data Studio settings and visualization for parking visits and percipitation -1-&lt;/figcaption&gt;
  &lt;/figure&gt;
&lt;figure&gt;
    &lt;img src=&#34;https://www.beginswithdata.com/post/2019-11-09/6g.png&#34; title=&#34;Data Studio settings and visualization for parking visits and percipitation -2-&#34; alt=&#34;Data Studio settings and visualization for parking visits and percipitation -2-&#34;  loading=&#34;lazy&#34; /&gt;
    &lt;figcaption&gt;Data Studio settings and visualization for parking visits and percipitation -2-&lt;/figcaption&gt;
  &lt;/figure&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;wrap-up&#34;&gt;Wrap up&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;wrap-up&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#wrap-up&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;In this short demo we setup a Petabyte scale data warehouse, loaded some data into it, queried and visualized the data, used native geospatial capabilities, and then combined our own data with a public dataset.  All this in the span of 30 minutes :-)
Also, although we used the BigQuery Sandbox, all you need to do is enable billing and keep going with the usage limits removed.
I hope you learned a little more about BigQuery and can adapt these steps to work with a dataset you are passionate about.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;for-more&#34;&gt;For more&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;for-more&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#for-more&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Check out some other BigQuery resources:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://towardsdatascience.com/bigquery-without-a-credit-card-discover-learn-and-share-199e08d4a064&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Use BigQuery free — without a credit card: Discover, learn and share&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://towardsdatascience.com/tagged/bigquery&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Towards Data Science - BigQuery Ta&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://cloud.google.com/blog/products/bigquery&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Google Cloud Blog - BigQuery Tag&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://www.reddit.com/r/bigquery/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;BigQuery Reddit&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Dynamic storage provisioning with OpenShift and NetApp</title>
      <link>https://www.beginswithdata.com/2017/07/13/2017-07-13-openshift-with-netapp/</link>
      <pubDate>Thu, 13 Jul 2017 19:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/07/13/2017-07-13-openshift-with-netapp/</guid>
      <description>
        
        
        &lt;p&gt;I love containers, and if I have to manage them at scale give me Kubernetes. Recently I started playing around with OpenShift, which is built on top of Kubernetes, and found even more to like.  For me, the ability to create &amp;lsquo;routes&amp;rsquo; (enabling external access to cluster services) and a GUI to see and influence what is happening, was already enough to show its value above the base Kubernetes.&lt;/p&gt;
&lt;p&gt;It all begins with data™ and bringing NetApp storage to the party was a top priority.  Fortunately, back in 2016 NetApp released a dynamic storage provisioner for OpenShift called Trident.  &lt;em&gt;IT IS AWESOME&lt;/em&gt;.  If you&amp;rsquo;re using OpenShift without dynamic storage provisioning you &lt;strong&gt;must check this out.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;But Trident doesn&amp;rsquo;t just automate storage management, it makes it autonomous, NoOps storage.  Ever observant, Trident watches the &lt;code&gt;kube-apiserver&lt;/code&gt; for outstanding &lt;code&gt;PersistentVolumeClaim&lt;/code&gt; requests submitted by users and quickly and swiftly chooses the backend storage system and provisions the volume, presents it to the nodes in the OpenShift Cluster, adds it as a &lt;code&gt;PersistentVolume&lt;/code&gt; in OpenShift, and binds it to the request.  All this happens before you can take a sip of your coffee allowing the application to start up fast.  Then when a user deletes the &lt;code&gt;PersistenVolumeClaim&lt;/code&gt; the reverse happens ensuring everything stays in sync and avoiding orphan, or wasted storage. I hate cleaning up, and having OpenShift and Trident take care of it for me saves me time and reduces risk.  And because Trident supports all the NetApp storage systems (SolidFire iSCSI, ONTAP iSCSI, ONTAP NFS, and E-Series iSCSI) I can use the right system for the right job without additional complexity.&lt;/p&gt;
&lt;p&gt;Curious?  I created a simple demo to show off a few of my favorite OpenShift + NetApp features:&lt;/p&gt;
&lt;iframe src=&#34;https://www.youtube.com/embed/WZ3nwl4aILU&#34; width=&#34;560&#34; height=&#34;315&#34; frameborder=&#34;0&#34; allowfullscreen=&#34;allowfullscreen&#34;&gt;&lt;/iframe&gt;
&lt;p&gt;Wanna try it yourself?  Here are some resources to get you on your way:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://github.com/NetApp/trident&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Trident GitHub: documentation and download&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://www.youtube.com/playlist?list=PLZdTVI9ql2pFNCfk1UuSeBIV7mxlkfz4J&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Trident YouTube: Concepts and installation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://netapp.io/2017/02/17/using-persistent-storage-openshift-applications-trident&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Trident blog: OpenShift with Trident&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Fixing filesystem corruption on a Kubernetes Persistent Volume</title>
      <link>https://www.beginswithdata.com/2017/06/23/2017-06-23-fsck-k8s/</link>
      <pubDate>Fri, 23 Jun 2017 16:05:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/06/23/2017-06-23-fsck-k8s/</guid>
      <description>
        
        
        &lt;p&gt;I like to break things.  Kubernetes is really good at maintaining desired state so I decided to play  &lt;a href=&#34;https://www.youtube.com/watch?v=GVJL9oXgsAA&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;whack-a-mole&lt;/a&gt; with my pod:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;# kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
etcd-2372957639-s3107   2/2       Running   1          1d

# kubectl delete pods etcd-2372957639-s3107
pod &amp;#34;etcd-2372957639-s3107&amp;#34; deleted

# kubectl get pods
NAME                       READY     STATUS              RESTARTS   AGE
etcd-2372957639-s3107   2/2       Terminating         1          1d
etcd-2372957639-tl5xh   0/2       ContainerCreating   0          4s&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;After dozens of &amp;lsquo;whacks&amp;rsquo; I noticed it got stuck with a status of &lt;code&gt;ContainerCreating&lt;/code&gt;.  Lets check the logs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;#&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubectl&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;logs&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;etcd&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-2372957639&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-tl&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;xh&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;etcd&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;Error from server (BadRequest): container &amp;#34;etcd&amp;#34; in pod &amp;#34;etcd-2372957639-tl5xh&amp;#34; is waiting to start: ContainerCreating&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Oh yeah, if the container hasn&amp;rsquo;t started yet it has no logs to view!  Lets describe the pod and check the events section:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;#&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubectl&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;describe&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pods&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;etcd&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-2372957639&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-tl&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;xh&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;...&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Events:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;FirstSeen&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;LastSeen&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;Count&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;From&lt;/span&gt;                    &lt;span class=&#34;err&#34;&gt;SubObjectPath&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;Type&lt;/span&gt;            &lt;span class=&#34;err&#34;&gt;Reason&lt;/span&gt;          &lt;span class=&#34;err&#34;&gt;Message&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;---------&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;--------&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;-----&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;----&lt;/span&gt;                    &lt;span class=&#34;err&#34;&gt;-------------&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;--------&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;------&lt;/span&gt;          &lt;span class=&#34;err&#34;&gt;-------&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;17&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;           &lt;span class=&#34;mi&#34;&gt;17&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;             &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;default-scheduler&lt;/span&gt;                       &lt;span class=&#34;err&#34;&gt;Normal&lt;/span&gt;          &lt;span class=&#34;err&#34;&gt;Scheduled&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;Successfully&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;assigned&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;etcd&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-2372957639&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-tl&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;xh&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;17&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;           &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;              &lt;span class=&#34;mi&#34;&gt;15&lt;/span&gt;      &lt;span class=&#34;err&#34;&gt;kubelet,&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;                      &lt;span class=&#34;err&#34;&gt;Warning&lt;/span&gt;         &lt;span class=&#34;err&#34;&gt;FailedMount&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;MountVolume.SetUp&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volume&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;s2&#34;&gt;&amp;#34;kubernetes.io/iscsi/fb6d5a06-56b5-11e7-bafb-525400f4fd4c-etcd&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(spec.Name:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;(UID:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;&amp;#39;fsck&amp;#39;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;found&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;device&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;but&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;could&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;not&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;correct&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;them:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;from&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;util-linux&lt;/span&gt; &lt;span class=&#34;mf&#34;&gt;2.23&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;contains&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;file&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;system&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;check&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;forced.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Unattached&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inode&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;23&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;UNEXPECTED&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;INCONSISTENCY;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;RUN&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;MANUALLY.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;(i.e.,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;without&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;or&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-p&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;options)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;             &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;              &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;kubelet,&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;                     &lt;span class=&#34;err&#34;&gt;Warning&lt;/span&gt;         &lt;span class=&#34;err&#34;&gt;FailedMount&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;MountVolume.SetUp&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volume&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;s2&#34;&gt;&amp;#34;kubernetes.io/iscsi/fb6d5a06-56b5-11e7-bafb-525400f4fd4c-etcd&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(spec.Name:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(UID:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;with:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;exit&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;status&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;32&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Mounting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;command:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Mounting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;arguments:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;ext&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;defaults&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Output:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;is&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;already&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;or&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;busy&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;15&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;            &lt;span class=&#34;mi&#34;&gt;24&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;s&lt;/span&gt;             &lt;span class=&#34;mi&#34;&gt;8&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;kubelet,&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;                     &lt;span class=&#34;err&#34;&gt;Warning&lt;/span&gt;         &lt;span class=&#34;err&#34;&gt;FailedMount&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;Unable&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt;  
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz_default(fb6d5a06-56b5-11e7-bafb-525400f4fd4c)&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;timeout&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;expired&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;waiting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;attach/mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;default&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;list&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;unattached/unmounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes=&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;etcd-vol&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;15&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;            &lt;span class=&#34;mi&#34;&gt;24&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;s&lt;/span&gt;             &lt;span class=&#34;mi&#34;&gt;8&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;kubelet,&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;                     &lt;span class=&#34;err&#34;&gt;Warning&lt;/span&gt;         &lt;span class=&#34;err&#34;&gt;FailedSync&lt;/span&gt;      &lt;span class=&#34;err&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;syncing&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;skipping:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;timeout&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;expired&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;waiting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;err&#34;&gt;attach/mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;default&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;list&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;unattached/unmounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes=&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;etcd-vol&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;So we can see the pod was assigned to the 3-docker2 host, but then when the kubelet tried to mount a volume the &lt;code&gt;ext4&lt;/code&gt; filesystem inside had errors that require manual intervention.  &lt;em&gt;&lt;strong&gt;Not good.&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Since the pod is scheduled to host 3-docker2 lets go there and check out the logs:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;#&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;journalctl&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-r&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-u&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;39.794177&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;iscsi_util.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;195&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;iscsi:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;iscsi&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volume&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;ext&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;error&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;&amp;#39;fsck&amp;#39;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;found&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;device&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;but&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;could&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;not&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;correct&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;them:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;from&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;util-linux&lt;/span&gt; &lt;span class=&#34;mf&#34;&gt;2.23&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;contains&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;file&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;system&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;check&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;forced.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Unattached&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inode&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;23&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;UNEXPECTED&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;INCONSISTENCY;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;RUN&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;MANUALLY.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(i.e.,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;without&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;or&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-p&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;options)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;39.794443&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;disk_manager.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;50&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;attach&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;disk&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;39.794471&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;iscsi.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;228&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;iscsi:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;setup&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;39.794898&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;nestedpendingoperations.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;262&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Operation&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;\&amp;#34;kubernetes.io/iscsi/fb6d5a06-56b5-11e7-bafb-525400f4fd4c-etcd\&amp;#34; (\&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c\&amp;#34;)&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed.&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;No&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;retries&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;permitted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;until&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;2017-06-21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;31&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;39.794803785&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;+&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0200&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;CEST&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(durationBeforeRetry&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;s).&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Error:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;MountVolume.SetUp&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volume&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;kubernetes.io/iscsi/fb6d5a06-56b5-11e7-bafb-525400f4fd4c-etcd&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(spec.Name:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(UID:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;&amp;#39;fsck&amp;#39;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;found&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;on&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;device&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;but&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;could&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;not&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;correct&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;them:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;from&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;util-linux&lt;/span&gt; &lt;span class=&#34;mf&#34;&gt;2.23&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;contains&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;file&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;system&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;check&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;forced.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Unattached&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inode&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;23&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;UNEXPECTED&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;INCONSISTENCY;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;RUN&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;MANUALLY.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(i.e.,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;without&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;or&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;-p&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;options)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;29&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;39&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;30&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;46&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;30&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;46.869197&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1549&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Unable&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz_default(fb6d5a06-56b5-11e7-bafb-525400f4fd4c)&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;timeout&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;expired&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;waiting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;attach/mount&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;default&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;list&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;unattached/unmounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes=&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;etcd-vol&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;skipping&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Jun&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;30&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;46&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubelet&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;E&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0621&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;21&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;30&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;46.869346&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;13812&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod_workers.go:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;182&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;]&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Error&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;syncing&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fb&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;6&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;d&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;a&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;06-56&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;b&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-11e7&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-bafb&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-525400&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;f&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;fd&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;c&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz_default(fb6d5a06-56b5-11e7-bafb-525400f4fd4c)&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;),&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;skipping:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;timeout&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;expired&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;waiting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;attach/mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;default&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd-2372957639-dkdwz&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;list&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;of&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;unattached/unmounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volumes=&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;etcd-vol&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;So basically the same story.&lt;/p&gt;
&lt;p&gt;The device is already presented to the host but just not mounted so run the &lt;code&gt;fsck&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;#&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;from&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;util-linux&lt;/span&gt; &lt;span class=&#34;mf&#34;&gt;2.23&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;e&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;fsck&lt;/span&gt; &lt;span class=&#34;mf&#34;&gt;1.42&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;9&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;28&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-Dec&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-2013&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/sda&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;contains&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;a&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;file&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;system&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;errors,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;check&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;forced.&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Pass&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Checking&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inodes,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;blocks,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;and&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;sizes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Pass&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Checking&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;directory&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;structure&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Pass&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Checking&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;directory&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;connectivity&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Pass&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Checking&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;reference&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;counts&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Unattached&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inode&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;23&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Connect&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;to&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/lost+found&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Inode&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;23&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;ref&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;count&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;is&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;should&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;be&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;  &lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Pass&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;Checking&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;group&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;summary&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;information&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Block&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;bitmap&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;differences:&lt;/span&gt;  &lt;span class=&#34;err&#34;&gt;+(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;7939&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-23563&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Free&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;blocks&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;count&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;wrong&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;group&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;#&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;16785&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;counted=&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;32410&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;).&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Free&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;blocks&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;count&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;wrong&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;391677&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;counted=&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;439712&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;).&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Inode&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;bitmap&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;differences:&lt;/span&gt;  &lt;span class=&#34;mi&#34;&gt;-17&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Free&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inodes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;count&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;wrong&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;group&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;#&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;8125&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;counted=&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;8126&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;).&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Free&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;inodes&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;count&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;wrong&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;122141&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;counted=&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;122142&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;).&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Fix&amp;lt;y&amp;gt;?&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;yes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;*****&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;FILE&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;SYSTEM&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;WAS&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;MODIFIED&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;*****&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;/dev/sda:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;18&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;122160&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;files&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;16.7&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;%&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;non-contiguous),&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;48736&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;488448&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;blocks&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;So we fixed it.  I&amp;rsquo;m not an expert on &lt;code&gt;ext4&lt;/code&gt; but there is probably a file in &lt;code&gt;lost+found&lt;/code&gt; we might want to check if we need.  In any case, the filesystem should be able to be mounted now.&lt;/p&gt;
&lt;p&gt;Lets give it a minute and check if Kubernetes got it up:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;#&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;kubectl&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;get&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pods&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;NAME&lt;/span&gt;                       &lt;span class=&#34;err&#34;&gt;READY&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;STATUS&lt;/span&gt;    &lt;span class=&#34;err&#34;&gt;RESTARTS&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;AGE&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;etcd&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-2372957639&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-tl&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;xh&lt;/span&gt;      &lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;Running&lt;/span&gt;   &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;          &lt;span class=&#34;mi&#34;&gt;50&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;There we go, our pod is running again!&lt;/p&gt;
&lt;p&gt;For the paranoid (me included) I was curious that no other admins or Kubernetes itself could corrupt things even more while I did my &lt;code&gt;fsck&lt;/code&gt;.  Checking the event log I see that while my &lt;code&gt;fsck&lt;/code&gt; was active it logged a new error because the device was busy:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;Events:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;FirstSeen&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;LastSeen&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;Count&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;From&lt;/span&gt;                    &lt;span class=&#34;err&#34;&gt;SubObjectPath&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;Type&lt;/span&gt;            &lt;span class=&#34;err&#34;&gt;Reason&lt;/span&gt;          &lt;span class=&#34;err&#34;&gt;Message&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;---------&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;--------&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;-----&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;----&lt;/span&gt;                    &lt;span class=&#34;err&#34;&gt;-------------&lt;/span&gt;   &lt;span class=&#34;err&#34;&gt;--------&lt;/span&gt;        &lt;span class=&#34;err&#34;&gt;------&lt;/span&gt;          &lt;span class=&#34;err&#34;&gt;-------&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;mi&#34;&gt;10&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;           &lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;m&lt;/span&gt;              &lt;span class=&#34;mi&#34;&gt;5&lt;/span&gt;       &lt;span class=&#34;err&#34;&gt;kubelet,&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;3&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-docker&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2&lt;/span&gt;      &lt;span class=&#34;err&#34;&gt;Warning&lt;/span&gt;         &lt;span class=&#34;err&#34;&gt;FailedMount&lt;/span&gt;     &lt;span class=&#34;err&#34;&gt;MountVolume.SetUp&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;volume&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;s2&#34;&gt;&amp;#34;kubernetes.io/iscsi/fb6d5a06-56b5-11e7-bafb-525400f4fd4c-etcd&amp;#34;&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;(spec.Name:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;etcd&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;pod&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;(UID:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;fb6d5a06-56b5-11e7-bafb-525400f4fd4c&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;with:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;failed:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;exit&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;status&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;32&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Mounting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;command:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Mounting&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;arguments:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/disk/by-path/ip&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;-192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iscsi-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.default-trash&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-7971&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;f.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2484&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;ext&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;4&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;defaults&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &lt;span class=&#34;err&#34;&gt;Output:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mount:&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/dev/sdh&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;is&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;already&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;mounted&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;or&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;192.168&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;mf&#34;&gt;1.190&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;3260&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-iqn.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;2010-01&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;.com.s:pnay.etcd-etcd.&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;1306&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;-lun&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;-0&lt;/span&gt; &lt;span class=&#34;err&#34;&gt;busy&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;A bit of filesystem recovery container style :-)&lt;/p&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Containerize everything:  Stateful apps on Kubernetes</title>
      <link>https://www.beginswithdata.com/2017/06/14/2017-06-14-techsummit-k8s-video/</link>
      <pubDate>Wed, 14 Jun 2017 08:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/06/14/2017-06-14-techsummit-k8s-video/</guid>
      <description>
        
        
        &lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2017-06-14/ready.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;In this 35 minute talk recorded at  &lt;a href=&#34;http://www.techsummit.io&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;TechSummit.io&lt;/a&gt; Amsterdam you will learn about deploying stateful apps in Docker containers on Kubernetes. First I will describe how app owners build their stateful apps using &lt;code&gt;StatefulSets&lt;/code&gt; and &lt;code&gt;PersistentVolumeClaims&lt;/code&gt;, and how these are underpinned by infrastructure resources like &lt;code&gt;PersistentVolumes&lt;/code&gt; and &lt;code&gt;StorageClasses&lt;/code&gt;.  If your stateful app is a single instance database these basics might be enough, but with a scale-out database a bit of application specific glue is needed to manage this scaling gracefully. A few design patterns are emerging to provide this glue and I will touch on a few of them.&lt;/p&gt;
&lt;p&gt;Included are two demos with persistent storage dynamically provisioned by &lt;a href=&#34;https://github.com/NetApp/trident&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Trident&lt;/a&gt; with Kubernetes v1.6.4:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Single instance WordPress + MySQL&lt;/li&gt;
&lt;li&gt;Scale-out with MongoDB&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I hope you like it!&lt;/p&gt;
&lt;iframe src=&#34;https://www.youtube.com/embed/UqVC0_BtNis&#34; width=&#34;560&#34; height=&#34;315&#34; frameborder=&#34;0&#34; allowfullscreen=&#34;allowfullscreen&#34;&gt;&lt;/iframe&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Using the NetApp Docker Volume Plugin with SolidFire storage</title>
      <link>https://www.beginswithdata.com/2017/02/06/ndvp-usage-solidfire-san/</link>
      <pubDate>Mon, 06 Feb 2017 17:00:00 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/02/06/ndvp-usage-solidfire-san/</guid>
      <description>
        
        
        &lt;p&gt;In this blog I will explain how to configure and use the &lt;a href=&#34;https://github.com/NetApp/netappdvp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Docker Volume Plugin (nDVP)&lt;/a&gt; with SolidFire storage.&lt;/p&gt;
&lt;p&gt;To get up and running follow these three main steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;a href=&#34;#1-install-the-ndvp-binary-on-your-container-host&#34;&gt;Install the nDVP binary on your container host&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#2-create-a-config-json-file&#34;&gt;Create a config json file&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#3-create-and-manage-volumes-from-docker&#34;&gt;Create and manage volumes from Docker&lt;/a&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;em&gt;and if it doesn&amp;rsquo;t work, check &lt;a href=&#34;#troubleshooting&#34;&gt;Troubleshooting&lt;/a&gt; :-)&lt;/em&gt;&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;1-install-the-ndvp-binary-on-your-container-host&#34;&gt;[1] Install the nDVP binary on your container host&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;1-install-the-ndvp-binary-on-your-container-host&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#1-install-the-ndvp-binary-on-your-container-host&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Install the nDVP software (a single binary) on your container host.  We will create the config json file and start the daemon in the next step, so for now just get the files on the container host:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Manual installation:
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://github.com/NetApp/netappdvp#quick-start&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Generic instructions&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://www.beginswithdata.com/2017/02/05/ndvp-install-coreos&#34;&gt;CoreOS instructions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Docker store installation (requires Docker v1.13 and greater.  Note Docker switched to a YYMM version scheme, so v17.X is greater than v1.13):
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;docker plugin install --alias netapp --grant-all-permissions --disable netapp/ndvp-plugin:17.04&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 data-hextra-search-id=&#34;2-create-a-config-json-file&#34;&gt;[2] Create a config json file&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;2-create-a-config-json-file&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#2-create-a-config-json-file&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;The nDVP uses a JSON formatted configuration file to learn connection information when it is started.  There is a 1:1 relationship between a storage array and a configuration file.  If you have multiple storage arrays you must configure a conf file, and start a nDVP daemon, for each one.&lt;/p&gt;
&lt;p&gt;SolidFire supports the following config file options:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th&gt;Option&lt;/th&gt;
					&lt;th&gt;Description&lt;/th&gt;
					&lt;th&gt;Example Value&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td&gt;version&lt;/td&gt;
					&lt;td&gt;Config file version number&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;1&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;storageDriverName&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;ontap-nas&lt;/code&gt;, &lt;code&gt;ontap-san&lt;/code&gt;, &lt;code&gt;eseries-iscsi&lt;/code&gt;, or &lt;code&gt;solidfire-san&lt;/code&gt;&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;solidfire-san&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;debug&lt;/td&gt;
					&lt;td&gt;Turn debugging output on or off&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;false&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;Endpoint&lt;/td&gt;
					&lt;td&gt;MVIF API endpoint formatted as : https://&lt;strong&gt;login&lt;/strong&gt;:&lt;strong&gt;password&lt;/strong&gt;@&lt;strong&gt;mvip&lt;/strong&gt;/json-rpc/&lt;strong&gt;element-version&lt;/strong&gt;&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;https://user:pw@10.9.8.7/json-rpc/7.0&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;SVIP&lt;/td&gt;
					&lt;td&gt;SVIP: iSCSI IP address and port&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;10.0.0.7:3260&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;TenantName&lt;/td&gt;
					&lt;td&gt;SF Tenant to use (created if not found)&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;docker&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;DefaultVolSz&lt;/td&gt;
					&lt;td&gt;Volume size in GiB&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;1&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;InitiatorIFace&lt;/td&gt;
					&lt;td&gt;Specify interface when restricting iSCSI traffic to non-default interface&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;default&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;Types&lt;/td&gt;
					&lt;td&gt;QoS specifications to simplify reuse of standard QoS settings&lt;/td&gt;
					&lt;td&gt;&lt;em&gt;See example below&lt;/em&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;LegacyNamePrefix&lt;/td&gt;
					&lt;td&gt;Prefix for upgraded NDVP installs to maintain mapping of volumes created with nDVP &amp;lt;1.3.2&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;netappdvp-&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Here is a working example to use as a template:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;version&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;storageDriverName&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;solidfire-san&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;debug&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;false&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;Endpoint&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;https://ndvpuser:not-my-real-pw@10.9.8.7/json-rpc/7.0&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;SVIP&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;10.0.0.7:3260&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;TenantName&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;docker&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;DefaultVolSz&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;InitiatorIFace&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;default&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;nt&#34;&gt;&amp;#34;Types&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;nt&#34;&gt;&amp;#34;Type&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;staging&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;nt&#34;&gt;&amp;#34;Qos&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;minIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;500&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;maxIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;4000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;burstIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;8000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;nt&#34;&gt;&amp;#34;Type&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;prod&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;nt&#34;&gt;&amp;#34;Qos&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;minIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;6000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;maxIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;8000&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;nt&#34;&gt;&amp;#34;burstIOPS&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;mi&#34;&gt;10000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Copy the working example from above into your favorite text editor.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;At a minimum, edit these fields:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;Endpoint&lt;/code&gt;:  Replace with your username:password and MVIF IP address&lt;/li&gt;
&lt;li&gt;&lt;code&gt;SVIP&lt;/code&gt;: Replace with SVIP IP address and iSCSI Port (normally 3260)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;TenantName&lt;/code&gt;: Replace with the Account name to use for this nDVP instance&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Write out the contents to &lt;code&gt;/etc/netappdvp/config.json&lt;/code&gt; on your container host:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;mkdir /etc/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat &lt;span class=&#34;s&#34;&gt;&amp;lt;&amp;lt; EOF &amp;gt; /etc/netappdvp/config.json
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;**PASTE YOUR JSON CONFIG CONTENTS HERE**
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s&#34;&gt;EOF&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Start the nDVP:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;If installed manually:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;/opt/netappdvp/netappdvp --config&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;/etc/netappdvp/config.json &lt;span class=&#34;p&#34;&gt;&amp;amp;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If installed from the Docker store:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker plugin &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; netapp&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;3-create-and-manage-volumes-from-docker&#34;&gt;[3] Create and manage volumes from Docker&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;3-create-and-manage-volumes-from-docker&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#3-create-and-manage-volumes-from-docker&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Volumes are managed using the &lt;code&gt;docker volume&lt;/code&gt; command family.  The examples all assume you started the nDVP using the default volume-driver name (&lt;code&gt;netapp&lt;/code&gt;).  Options can be passed to volume plugins using one or more &lt;code&gt;-o KEY=VALUE&lt;/code&gt; arguments.&lt;/p&gt;
&lt;p&gt;Supported options for SolidFire iSCSI are:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Option&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Description&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Default&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Example&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;size&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Size in GiB.&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;DefaultVolSz&lt;/code&gt; from nDVP json config file&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;115&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;type&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;QoS Types entry from nDVP json config file&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;SolidFire array default&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;gold&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;qos&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;MIN,MAX,BURST QoS IOP values&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;SolidFire array default&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;100,500,1500&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;from&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Create a clone from this volume&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;em&gt;none&lt;/em&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;vol1&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;from-snapshot&lt;/code&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;Snapshot name to use for clone&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;em&gt;none&lt;/em&gt;&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;snap1&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;SolidFire iSCSI volume &lt;strong&gt;create&lt;/strong&gt; examples:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a volume using default settings for size and QoS&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name volDefault
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a 65GB volume using default QoS settings&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name vol65 -o &lt;span class=&#34;nv&#34;&gt;size&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;65&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a 75GB volume using QoS settings from the Gold Type&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name volGold -o &lt;span class=&#34;nv&#34;&gt;size&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;75&lt;/span&gt; -o &lt;span class=&#34;nv&#34;&gt;type&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;gold
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a 85GB volume using QoS of Min:100, Max:200, Burst:1000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name volQos -o &lt;span class=&#34;nv&#34;&gt;size&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;&lt;span class=&#34;m&#34;&gt;85&lt;/span&gt; -o &lt;span class=&#34;nv&#34;&gt;qos&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;100,200,1000
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a clone volume from an existing volume volGold (QoS settings are inherited)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name volGoldClone -o &lt;span class=&#34;nv&#34;&gt;from&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;volGold
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Create a clone volume from an existing volume volGold, snapshot upgrade55 (QoS settings are inherited)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume create --driver netapp --name volGoldClone -o &lt;span class=&#34;nv&#34;&gt;from&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;volGold -o from-snapshot&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;upgrade55&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Volume &lt;strong&gt;list&lt;/strong&gt; examples:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# List all volumes&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume ls
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# List volumes for driver netapp&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume ls -f &lt;span class=&#34;nv&#34;&gt;driver&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;netapp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Inspect (full listing of) a specific volume.  Note that snapshot names are shown&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# which is helpful when creating a clone using the from-snapshot option&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume inspect volGold
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;o&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Name&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;volGold&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Driver&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;solidfire-san&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Mountpoint&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Status&amp;#34;&lt;/span&gt;: &lt;span class=&#34;o&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;s2&#34;&gt;&amp;#34;Snapshots&amp;#34;&lt;/span&gt;: &lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;o&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                    &lt;span class=&#34;s2&#34;&gt;&amp;#34;Created&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;2017-02-03T20:14:32Z&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                    &lt;span class=&#34;s2&#34;&gt;&amp;#34;Name&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;maddenSnap&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                &lt;span class=&#34;o&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;            &lt;span class=&#34;o&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;o&#34;&gt;}&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Labels&amp;#34;&lt;/span&gt;: &lt;span class=&#34;o&#34;&gt;{}&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        &lt;span class=&#34;s2&#34;&gt;&amp;#34;Scope&amp;#34;&lt;/span&gt;: &lt;span class=&#34;s2&#34;&gt;&amp;#34;global&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    &lt;span class=&#34;o&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Volume &lt;strong&gt;remove&lt;/strong&gt; example:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Remove volume named volGoldClone; will remove the Docker volume and the volume&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# from the SolidFire array&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker volume rm volGoldClone&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Attaching a volume to a container providing &lt;strong&gt;persistent&lt;/strong&gt; storage:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Run a Debian bash shell and attach gold-vol to /gold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker run -it --rm -v volGold:/gold debian bash
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Write something to the volume and exit&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;echo&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;SolidFire: zero touch storage&amp;#34;&lt;/span&gt; &amp;gt;&amp;gt; /gold/zero.txt
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;exit&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Run a CentOS bash shell and attach gold-vol to /gold&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker run -it --rm -v volGold:/gold centos bash
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Read from the persistent disk and exit&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat /gold/zero.txt
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;exit&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;h2 data-hextra-search-id=&#34;putting-it-all-together&#34;&gt;Putting it all together&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;putting-it-all-together&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#putting-it-all-together&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Seeing the input/output helps sometimes so check out this Asciicast [which allows copy/paste!] for a demo of the above!&lt;/p&gt;
&lt;script type=&#34;text/javascript&#34; src=&#34;https://asciinema.org/a/100240.js&#34; id=&#34;asciicast-100240&#34; async&gt;&lt;/script&gt;
&lt;h2 data-hextra-search-id=&#34;troubleshooting&#34;&gt;Troubleshooting&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;troubleshooting&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#troubleshooting&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;The nDVP daemon must be running for it to manage NetApp storage.  Check that it is running with:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;ps -ef &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; grep netappdvp&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Check the nDVP logfile for status or error messages:&lt;/p&gt;
&lt;p&gt;If installed manually:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;tail /var/log/netappdvp/*&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;If installed using &lt;code&gt;docker plugin&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# find the container&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker-runc list
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# view the logs in the container&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;docker-runc &lt;span class=&#34;nb&#34;&gt;exec&lt;/span&gt; -t &amp;lt;container id&amp;gt; cat /var/log/netappdvp/netapp.log&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Installing the NetApp Docker Volume Plugin on CoreOS</title>
      <link>https://www.beginswithdata.com/2017/02/05/ndvp-install-coreos/</link>
      <pubDate>Sun, 05 Feb 2017 17:00:00 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/02/05/ndvp-install-coreos/</guid>
      <description>
        
        
        &lt;p&gt;In this blog I will provide the steps to install the NetApp Docker Volume Plugin (nDVP) on CoreOS &lt;a href=&#34;#install-ndvp-manually&#34;&gt;manually&lt;/a&gt; and &lt;a href=&#34;#install-ndvp-with-cloud-config&#34;&gt;automatically&lt;/a&gt; with cloud-config.&lt;/p&gt;
&lt;p&gt;First, some introductions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://github.com/NetApp/netappdvp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Docker Volume Plugin (nDVP)&lt;/a&gt;:
&lt;ul&gt;
&lt;li&gt;Provides direct integration with the Docker ecosystem for NetApp storage including ONTAP, E-Series, and SolidFire.  The nDVP allows for Docker integrated storage management including provisioning/deprovisioning, attaching/detaching, and cloning of NetApp storage.  The nDVP software (a single binary) must be installed on the container host and reads its configuration from a JSON formatted config file when it is started. Each config file describes the connection with a single storage array config; multiple arrays == multiple nDVP daemons, each with a unique config file.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://www.coreos.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CoreOS&lt;/a&gt;:
&lt;ul&gt;
&lt;li&gt;A barebones linux distribution designed specifically to run containers at scale.  Some common linux features like the OS installer and a package manager are absent.  On the other hand, container ecosystem software like docker, rkt, and etcd are pre-installed.  CoreOS is also self-updating and does so for all components in an atomic manner using an A/B partition strategy which causes the &lt;code&gt;/usr&lt;/code&gt; filesystem to be read-only.  Given CoreOS is a bit unique in how it works installing software on it needs some special care.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 data-hextra-search-id=&#34;install-ndvp-manually&#34;&gt;Install nDVP manually&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;install-ndvp-manually&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#install-ndvp-manually&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;To install manually follow these steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Become root:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;sudo -i&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you will use iSCSI connected storage start the iSCSI service:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl start iscsid&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a nDVP configuration file for your storage array type.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Copy a template from &lt;a href=&#34;https://github.com/NetApp/netappdvp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;nDVP Github site&lt;/a&gt; for your platform and customize to match your site details.&lt;/li&gt;
&lt;li&gt;Validate the JSON using &lt;a href=&#34;http://jsonlint.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JSONlint&lt;/a&gt; to avoid a syntax error that will prevent the driver from starting&lt;/li&gt;
&lt;li&gt;Save this file to your container host. There are no requirements about name or directory location but for standardization the following filenames (matching the driver name) are suggested:
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Storage Driver&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Full Path to Config file&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;SolidFire iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-solidfire-san.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;ONTAP NFS&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-ontap-nas.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;ONTAP iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-ontap-san.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;E-Series iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-eseries-iscsi.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Download, install, and start the plugin.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Replace the FULL-PATH-TO-CONFIG-FILENAME with the name of your config file&lt;/li&gt;
&lt;li&gt;Replace the version of netappdvp with your desired version&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Make a directory for the nDVP&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;mkdir -p /opt/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Download and install the software&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget https://github.com/NetApp/netappdvp/releases/download/v1.3.3/netappdvp-1.3.3.tar.gz -O /tmp/netappdvp-1.3.3.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;tar -zxvf /tmp/netappdvp-1.3.3.tar.gz -C /opt/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;chown root:root /opt/netappdvp/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;chmod &lt;span class=&#34;m&#34;&gt;755&lt;/span&gt; /opt/netappdvp/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# Start the nDVP passing the full path to your config file, and put it to the background&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;/opt/netappdvp/netappdvp --config&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;CONFIG-FILENAME &lt;span class=&#34;p&#34;&gt;&amp;amp;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;You should now be able to use &lt;code&gt;docker volume COMMAND -d netapp&lt;/code&gt; commands to manage persistent storage.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;install-ndvp-with-cloud-config&#34;&gt;Install nDVP with cloud-config&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;install-ndvp-with-cloud-config&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#install-ndvp-with-cloud-config&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;To install using &lt;code&gt;cloud-config&lt;/code&gt; you need to add a few new snippets.  These instructions assume you will distribute using a &lt;a href=&#34;https://coreos.com/os/docs/1298.1.0/config-drive.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Config Drive&lt;/a&gt; and a &lt;code&gt;user_data&lt;/code&gt; file:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Create a &lt;a href=&#34;https://coreos.com/os/docs/latest/cloud-config.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;cloud-config&lt;/a&gt; file named &lt;code&gt;user_data&lt;/code&gt; in the current working directory.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a nDVP configuration file for your storage array type.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Copy a template from &lt;a href=&#34;https://github.com/NetApp/netappdvp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;nDVP Github site&lt;/a&gt; for your platform into a text editor and customize to match your site details.  It will be added to the &lt;code&gt;cloud-config&lt;/code&gt; in the next step.&lt;/li&gt;
&lt;li&gt;Validate the JSON using &lt;a href=&#34;http://jsonlint.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;JSONlint&lt;/a&gt; to avoid a syntax error that will prevent the driver from starting&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add a &lt;code&gt;write_files&lt;/code&gt; definition to &lt;code&gt;user_data&lt;/code&gt; so your nDVP config file will be saved on the container host:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;There are no requirements about name or directory location but for standardization the following filenames (matching the driver name) are suggested:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Storage Driver&lt;/th&gt;
					&lt;th style=&#34;text-align: left&#34;&gt;Full Path to Config file&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;SolidFire iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-solidfire-san.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;ONTAP NFS&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-ontap-nas.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;ONTAP iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-ontap-san.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;E-Series iSCSI&lt;/td&gt;
					&lt;td style=&#34;text-align: left&#34;&gt;&lt;code&gt;/etc/netappdvp-eseries-iscsi.json&lt;/code&gt;&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Replace FULL-PATH-TO-CONFIG-FILENAME with the name of your nDVP config file including the full path (example: &lt;code&gt;/etc/netappdvp-solidfire-san.json&lt;/code&gt;)&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c&#34;&gt;# Write out nDVP config files&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;write_files&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;path&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;FULL-PATH-TO-CONFIG-FILENAME&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;permissions&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;0644&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;owner&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;root&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;content&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;sd&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      # Paste your nDVP config JSON snippet here; indentation in YAML matters!&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you will use iSCSI connected storage start the iSCSI service by adding a new unit definition to your &lt;code&gt;user_data&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;coreos&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;units&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;c&#34;&gt;# Start iSCSI&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;iscsid.service&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;command&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;l&#34;&gt;start&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add a unit definition to download, install, and start the nDVP.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Remove the &lt;code&gt;coreos:&lt;/code&gt; and &lt;code&gt;units:&lt;/code&gt; lines if you already have this header.&lt;/li&gt;
&lt;li&gt;Remove the &lt;code&gt;After=iscsid.service&lt;/code&gt; line if you are not using iSCSI&lt;/li&gt;
&lt;li&gt;Replace FULL-PATH-TO-CONFIG-FILENAME with the name of your nDVP config file including the full path (example: &lt;code&gt;/etc/netappdvp-solidfire-san.json&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Replace the version of netappdvp with your desired version&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nt&#34;&gt;coreos&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;  &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;units&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;c&#34;&gt;# Install and start ndvp service&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;- &lt;span class=&#34;nt&#34;&gt;name&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;ndvp.service&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;      &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;command&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;start&amp;#34;&lt;/span&gt;&lt;span class=&#34;w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;nt&#34;&gt;content&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;p&#34;&gt;|&lt;/span&gt;&lt;span class=&#34;sd&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      [Unit]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      Description=nDVP service
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      After=iscsid.service
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      [Service]
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      Restart=always
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      ExecStartPre=-/usr/bin/mkdir -p /opt/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      ExecStartPre=-/usr/bin/wget https://github.com/NetApp/netappdvp/releases/download/v1.3.3/netappdvp-1.3.3.tar.gz -O /tmp/netappdvp-1.3.3.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      ExecStartPre=-/usr/bin/tar -zxvf /tmp/netappdvp-1.3.3.tar.gz -C /opt/netappdvp
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;sd&#34;&gt;      ExecStart=/opt/netappdvp/netappdvp --config=FULL-PATH-TO-CONFIG-FILENAME&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Deploy!  You should be able to use &lt;code&gt;docker volume COMMAND -d netapp&lt;/code&gt; commands to manage persistent storage.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;troubleshooting&#34;&gt;Troubleshooting&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;troubleshooting&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#troubleshooting&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;The nDVP daemon must be running for it to manage NetApp storage.  Check that it is running with:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;ps -ef &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; grep netappdvp&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Check the nDVP logfile for status or error messages:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;tail /var/log/netappdvp/*&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enable debug logging, try the action again, and then check the log.  Enable debug logging in one of two ways:&lt;/p&gt;
&lt;p&gt;(a) Edit your nDVP config JSON to include and then restart the nDVP:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;debug&amp;#34;&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;kc&#34;&gt;true&lt;/span&gt;&lt;span class=&#34;err&#34;&gt;,&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;or (b) [Re]start the netappdvp with the &lt;code&gt;-debug&lt;/code&gt; flag:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;/opt/netappdvp/netappdvp --config&lt;span class=&#34;o&#34;&gt;=&lt;/span&gt;FULL-PATH-TO-CONFIG-FILENAME  -debug&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Docker Swarm on Container Lab demo</title>
      <link>https://www.beginswithdata.com/2017/01/02/docker-swarm-on-container-lab/</link>
      <pubDate>Mon, 02 Jan 2017 01:03:55 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2017/01/02/docker-swarm-on-container-lab/</guid>
      <description>
        
        
        &lt;p&gt;If you used &lt;a href=&#34;https://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/&#34;&gt;Using CentOS 7, KVM, and CoreOS to build a container lab&lt;/a&gt; and have a KVM host or two ready to go here is your first project: Docker Swarm.&lt;/p&gt;
&lt;p&gt;In this 3 minute long screencast we will:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Build three CoreOS container hosts&lt;/li&gt;
&lt;li&gt;Create a 3 node swarm (1 manager, 2 workers) on them&lt;/li&gt;
&lt;li&gt;Create a web app with 3 replicas&lt;/li&gt;
&lt;li&gt;Verify we can access the web app and it is load balanced by Swarm&lt;/li&gt;
&lt;li&gt;Scale up to 15 replicas&lt;/li&gt;
&lt;li&gt;Simulate failures by destroying the 2 worker nodes and verify the web app stays up&lt;/li&gt;
&lt;li&gt;Remove the last node as final clean up&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The ascii screencast has no audio but allows copy/paste.  Depending on your screen size you might want to go full screen to avoid missing some content.&lt;/p&gt;
&lt;p&gt;Without further delay, here it is:&lt;/p&gt;
&lt;script type=&#34;text/javascript&#34; src=&#34;https://asciinema.org/a/2ca9qxz7s1zzavchrohso3g8r.js&#34; data-speed=&#34;2&#34;  id=&#34;asciicast-2ca9qxz7s1zzavchrohso3g8r&#34; async&gt;&lt;/script&gt;
&lt;p&gt;As always, comments are welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Using CentOS 7, KVM, and CoreOS to build a container lab</title>
      <link>https://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/</link>
      <pubDate>Fri, 30 Dec 2016 16:41:57 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/</guid>
      <description>
        
        
        &lt;p&gt;Containers are the next big thing.  &lt;strong&gt;Bigger than Virtual Machines&lt;/strong&gt;.  No question about it.&lt;/p&gt;
&lt;p&gt;To learn about containers, Docker is the logical place to start. &lt;a href=&#34;https://docs.docker.com/machine/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Docker Machine&lt;/a&gt; provides a Docker engine based  environment that is point and click simple for your PC.  It&amp;rsquo;s awesome. This is your first stop to get a taste.&lt;/p&gt;
&lt;p&gt;Soon you&amp;rsquo;ll want more.  Lots more.  Your list of &amp;rsquo;things to check out&amp;rsquo; will start to grow:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Optimized Container Host OS: &lt;a href=&#34;https://coreos.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CoreOS&lt;/a&gt;, &lt;a href=&#34;https://www.projectatomic.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Atomic Host&lt;/a&gt;, &lt;a href=&#34;https://www.ubuntu.com/core&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ubuntu Core&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Orchestrators: &lt;a href=&#34;http://kubernetes.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kubernetes&lt;/a&gt;, &lt;a href=&#34;https://www.docker.com/products/docker-swarm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Docker Swarm&lt;/a&gt;, &lt;a href=&#34;https://mesosphere.github.io/marathon/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mesos and Marathon&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Platform as a Service: &lt;a href=&#34;https://www.openshift.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;OpenShift&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Volume management: &lt;a href=&#34;https://github.com/NetApp/netappdvp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Docker Volume Plugin&lt;/a&gt;, &lt;a href=&#34;https://github.com/NetApp/trident&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Kubernetes volume provisioner &amp;lsquo;Trident&amp;rsquo;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Infra: &lt;a href=&#34;https://coreos.com/etcd/docs/latest/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;etcd&lt;/a&gt;, &lt;a href=&#34;https://coreos.com/flannel/docs/latest/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;flannel&lt;/a&gt;, &lt;a href=&#34;https://www.puppet.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;puppet&lt;/a&gt;, &lt;a href=&#34;https://www.chef.io&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;chef&lt;/a&gt;, &lt;a href=&#34;https://www.ansible.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ansible&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Container engines: &lt;a href=&#34;https://www.docker.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Docker&lt;/a&gt;, &lt;a href=&#34;https://coreos.com/rkt/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Rkt&lt;/a&gt;, &lt;a href=&#34;https://clearlinux.org/container&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intel Clear Containers&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;and the list goes on&amp;hellip;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The container ecosystem also moves fast.  Keeping up with all this is nearly impossible, and unless you have a streamlined way to create/destroy container hosts there isn&amp;rsquo;t enough time in the day to do it.  You could use the cloud, but this has other considerations (and costs) to it.&lt;/p&gt;
&lt;p&gt;In this blog you will find instructions on how to create a small container lab / sandbox using a one or more &lt;em&gt;retired&lt;/em&gt; production servers to allow you to play in a streamlined way.&lt;/p&gt;
&lt;p&gt;The tech starts with CentOS 7 as the hypervisor host OS, then adds QEMU/KVM/libvirt on top, and finally CoreOS VMs with their native Docker as container hosts. Lets get started!&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;prepare&#34;&gt;Prepare&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;prepare&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#prepare&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ol&gt;
&lt;li&gt;Find some old physical servers.&lt;/li&gt;
&lt;li&gt;Reserve some IP addresses, for simplicity in the same /24 network:
&lt;ul&gt;
&lt;li&gt;One for each of your CentOS 7 physical servers (1-5)&lt;/li&gt;
&lt;li&gt;One for each of your CoreOS VMs (20+)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Create DNS entries for those IP addresses, for hypervisor hosts like kvm1, kvm2, kvm3 and for container hosts like docker1, docker2, docker3, to keep connections simple.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;install-centos-7-as-our-kvm-hosts&#34;&gt;Install CentOS 7 as our KVM hosts&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;install-centos-7-as-our-kvm-hosts&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#install-centos-7-as-our-kvm-hosts&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;I installed using a USB stick and my Windows PC:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Download the &lt;a href=&#34;https://www.centos.org/download/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;Minimal ISO&amp;rdquo;&lt;/a&gt; to your PC.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Download &lt;a href=&#34;https://sourceforge.net/projects/win32diskimager/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Win32DiskImager&lt;/a&gt;. Run the  program and select the ISO file and the USB device drive letter and click Write:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/win32diskimager.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Stick the USB in a free port of your server and boot it.  You will probably have to interrupt the boot at some point to select the USB drive as your boot device.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Install CentOS normal.  Most things you can leave with the default.  The most important places to configure are networking, the installation location, timezone, and to set user credentials.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;For &lt;strong&gt;networking&lt;/strong&gt; click the config icon from the home screen.  First enter your hostname.  Next select the NIC and then toggle the slider on the right side from OFF to ON so the interface will be UP at boot.  Next click the configure button, go to the IP v4 tab, select Method: Manual, click add to enter IP details, then DNS server(s), and DNS search domains.  Then save it.&lt;/li&gt;
&lt;li&gt;For &lt;strong&gt;storage&lt;/strong&gt; you need to simply select the device and (automatic partitioning) and hit Done.  If you get an error that not enough space is available you can &amp;lsquo;reclaim space&amp;rsquo; which is essentially destroying existing partitions that were discovered by the installer.  Since you want to reuse this server you can reclaim all partitions which should give the installer space to continue&lt;/li&gt;
&lt;li&gt;For &lt;strong&gt;timezone&lt;/strong&gt; simply pick a city that is appropriate for your location&lt;/li&gt;
&lt;li&gt;Enter &lt;strong&gt;passwords&lt;/strong&gt;.  After you click &amp;ldquo;Begin Installation&amp;rdquo; and software is installing you can set credentials for two user accounts.  Do it!&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;After a few minutes you will see an installation successful message.  Remove the USB drive, reboot, and the server should come up and be reachable on the IP address and credentials you set.  Login via SSH as root (or become root using &lt;code&gt;sudo -i&lt;/code&gt;) and then continue.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Setup SSH authentication so you can ssh commands between any CentOS KVM host without typing a password and never hit issues with SSH host keys changed (as can happen in a lab as you re-use IPs).  This setup is NOT appropriate for production!&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Create a new SSH key to be used for all KVM host access&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;ssh-keygen -t rsa -b &lt;span class=&#34;m&#34;&gt;4096&lt;/span&gt; -C &lt;span class=&#34;s2&#34;&gt;&amp;#34;kvm&amp;#34;&lt;/span&gt; -f ~/.ssh/id_rsa -N &lt;span class=&#34;s2&#34;&gt;&amp;#34;&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add the public key as an authorized key:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cat ~/.ssh/id_rsa.pub &amp;gt;&amp;gt; ~/.ssh/authorized_keys
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;chmod &lt;span class=&#34;m&#34;&gt;0600&lt;/span&gt; ~/.ssh/authorized_keys&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Disable host key checking and use of a known_hosts file:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;echo&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;StrictHostKeyChecking no&amp;#34;&lt;/span&gt; &amp;gt;&amp;gt; ~/.ssh/config
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;echo&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;UserKnownHostsFile /dev/null&amp;#34;&lt;/span&gt; &amp;gt;&amp;gt; ~/.ssh/config&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Lastly, ssh to each other KVM host and copy the SSH config to it:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;mkdir ~/.ssh&lt;span class=&#34;p&#34;&gt;;&lt;/span&gt; chmod &lt;span class=&#34;m&#34;&gt;0700&lt;/span&gt; ~/.ssh&lt;span class=&#34;p&#34;&gt;;&lt;/span&gt; scp INSERT-KVM-IP-HERE:~/.ssh/* ~/.ssh/.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Update all the installed packages to the latest levels:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;yum -y update&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Stop and disable the default firewall, disable selinux, and reboot to make it all active to simply operation in a lab:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;sed -i --follow-symlinks &lt;span class=&#34;s1&#34;&gt;&amp;#39;s/^SELINUX=.*/SELINUX=disabled/g&amp;#39;&lt;/span&gt; /etc/sysconfig/selinux
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl stop firewalld
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl disable firewalld
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;reboot&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;At this point you have your CentOS host ready and need to install KVM on top.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;load-qemukvmlibvirtkimchi-on-centos-7&#34;&gt;Load QEMU/KVM/libvirt/kimchi on CentOS 7&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;load-qemukvmlibvirtkimchi-on-centos-7&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#load-qemukvmlibvirtkimchi-on-centos-7&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;To be able to run our CoreOS container host VMs on top of CentOS 7 system we will use:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://wiki.qemu.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;QEMU&lt;/a&gt;: Machine emulator and virtualizer&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.linux-kvm.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;KVM&lt;/a&gt;: Enables use of virtualization extensions provided by Intel and AMD processors&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://libvirt.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;libvirt&lt;/a&gt;: A Toolkit to manage virtualization hosts&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://kimchi-project.github.io/kimchi/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;kimchi&lt;/a&gt;: A lightweight web GUI for your host and KVM VMs&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Follow these steps:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Install a bunch of packages:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;yum -y install qemu-kvm libvirt virt-install bridge-utils bind-utils &lt;span class=&#34;se&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virt-manager wget net-tools virt-viewer genisoimage epel-release&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Start libvirt and enable it for autostart:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl start libvirtd
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; libvirtd&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Verify your installation:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;root@3-kvm3 ~&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;c1&#34;&gt;# virsh nodeinfo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;CPU model:           x86_64
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;CPU&lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;s&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;:              &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;CPU frequency:       &lt;span class=&#34;m&#34;&gt;2333&lt;/span&gt; MHz
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;CPU socket&lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;s&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;:       &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Core&lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;s&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt; per socket:  &lt;span class=&#34;m&#34;&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Thread&lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;s&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt; per core:  &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;NUMA cell&lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;s&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;:        &lt;span class=&#34;m&#34;&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Memory size:         &lt;span class=&#34;m&#34;&gt;4192208&lt;/span&gt; KiB&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;And that you &lt;strong&gt;don&amp;rsquo;t&lt;/strong&gt; have a message like this, in which case you need to enable virtualization extensions in the hardware BIOS:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;root@3-kvm3 ~&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt;&lt;span class=&#34;c1&#34;&gt;# dmesg | grep kvm&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;    0.745756&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt; systemd&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;1&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt;: Set hostname to &amp;lt;3-kvm3&amp;gt;.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;o&#34;&gt;[&lt;/span&gt;   11.851827&lt;span class=&#34;o&#34;&gt;]&lt;/span&gt; kvm: disabled by bios&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enable bridge networking so that your container host VMs can have routable IPs on the same network as the KVM host.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Identify the linux interface name for your active network connection using &lt;code&gt;ip a&lt;/code&gt;.  Change directory into the &lt;code&gt;networking-scripts&lt;/code&gt; directory and then copy the config file for your active network connection to a new bridge config file:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/ip-address.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Edit the &lt;code&gt;ifcfg-bridge0&lt;/code&gt; as follows (left is original, right is modified):&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/ip-bridge.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Edit the &lt;code&gt;ifcfg-YOUR-INT-NAME&lt;/code&gt; as follows (left is original, right is modified):&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/ip-ethernet.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Finally, restart networking and verify your fixed IP address is now assigned to the bridge interface, and that the physical interface is providing access to the bridge:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/ip-restart.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Install kimchi and set the web UI to auto-logout after 24hrs of inactivity:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;cd&lt;/span&gt; /tmp
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget http://kimchi-project.github.io/wok/downloads/latest/wok.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget http://kimchi-project.github.io/gingerbase/downloads/latest/ginger-base.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget http://kimchi-project.github.io/kimchi/downloads/latest/kimchi.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;yum -y install wok.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;yum -y install ginger-base.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;yum -y install kimchi.el7.centos.noarch.rpm
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;sed -i  &lt;span class=&#34;s1&#34;&gt;&amp;#39;s/^#session_timeout = .*/session_timeout = 1440/g&amp;#39;&lt;/span&gt; /etc/wok/wok.conf
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl &lt;span class=&#34;nb&#34;&gt;enable&lt;/span&gt; wokd
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;systemctl start wokd&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Go in your web browser to https://INSERT-KVM-IP-HERE:8001 and you will see a login page. Use your linux root credentials to login.  Then click the Host tab and you should see something like this:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/kimchi-host.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;prepare-and-install-coreos-on-top-of-kvm&#34;&gt;Prepare and install CoreOS on top of KVM&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;prepare-and-install-coreos-on-top-of-kvm&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#prepare-and-install-coreos-on-top-of-kvm&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;CoreOS is a barebones linux distribution designed specifically to run containers at scale.  Some common linux features like the OS installer and a package manager are absent.  On the other hand, container ecosystem software like docker, rkt, and etcd are included.&lt;/p&gt;
&lt;p&gt;CoreOS is self-updating and does so for all components in an atomic manner using an A/B partition strategy:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;If CoreOS is booted from the A partition and a software update becomes available it will be automatically loaded onto the B partition and set as active.&lt;/li&gt;
&lt;li&gt;When the host is rebooted it uses the B partition and when a future software update becomes available it would be loaded on to the A partition.&lt;/li&gt;
&lt;li&gt;The A/B partition is mounted as the /usr filesystem and is read-only.  Data stored outside of /usr will be persistent.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;CoreOS uses &lt;a href=&#34;https://coreos.com/os/docs/latest/cloud-config.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;cloud-config&lt;/a&gt; to set configuration details like user accounts, ssh keys, hostnames, networking configuration, and more, at every boot.  This file can be loaded from a filesystem location, a mounted CDROM image, or fetched from a URL.  The instructions on the CoreOS website for libvirt use a filesystem location pass-through by KVM (aka p9) but this feature is not available on CentOS so an extra step using an ISO was needed.&lt;/p&gt;
&lt;p&gt;Follow these steps to setup CoreOS:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;SSH into one of your KVM hosts.  Create or attach networked storage for the VM images, or use local disk and hope the IOPs are enough!  For these instructions I used &lt;code&gt;/vm-storage&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Download CoreOS QEMU image:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;mkdir -p /vm-storage/coreos
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;cd&lt;/span&gt; /vm-storage/coreos
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget https://stable.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2 -O - &lt;span class=&#34;p&#34;&gt;|&lt;/span&gt; bzcat &amp;gt; os.img&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Create a cloud-config file using a template I created. Download  &lt;a href=&#34;https://gist.github.com/dutchiechris/e5689a526d4651f6dec4880230fe3c89&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;user_data&lt;/a&gt; and save in &lt;code&gt;/vm-storage/coreos/user_data&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;cd&lt;/span&gt; /vm-storage/coreos
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget https://gist.githubusercontent.com/dutchiechris/e5689a526d4651f6dec4880230fe3c89/raw/4576b956ab00430e644917f17080f4ccfd408615/user_data
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;vi user_data&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Edit the file as described below:&lt;/p&gt;
&lt;script src=&#34;https://gist.github.com/dutchiechris/e5689a526d4651f6dec4880230fe3c89.js&#34;&gt;&lt;/script&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Line 1&lt;/strong&gt;: Must be &lt;code&gt;#cloud-config&lt;/code&gt; to be parsed&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Line 3-5&lt;/strong&gt;: SSH authorized_keys configuration.  Update line 4 and 5 to include your public key(s); see step 4 from the CentOS installation session for how to find or create these.  You can re-use the ones from your CentOS hosts, create another specifically for your container hosts, or do both like I did.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Line 7&lt;/strong&gt;: Sets the hostname.  The &lt;code&gt;@@VMNAME@@&lt;/code&gt; is a placeholder and will be customized by a startup script to be explained later; leave this one as-is.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Line 9-11&lt;/strong&gt;: Set a password for the &lt;code&gt;core&lt;/code&gt; user account to enable password authentication in addition to the SSH key authentication.  Update the password on line 11 with a password generated using &lt;code&gt;python -c &#39;import crypt,getpass; print(crypt.crypt(getpass.getpass(), crypt.mksalt(crypt.METHOD_SHA512)))&#39;&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Line 14-15&lt;/strong&gt;: Defines the update strategy and disables auto-reboot of CoreOS after it updates itself to avoid any surprise reboots&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Line 16-37&lt;/strong&gt;: Sets fixed IP configuration and disables DHCP.  On line 28 the &lt;code&gt;@@IP@@&lt;/code&gt; is a placeholder and will be customized by a startup script to be explained later; leave this one as-is.  Modify line 29 and 30 to fit your lab IP network range.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Using the CoreOS image and cloud-config from above I wrote a script to ease roll-out of many copies of these.  Download  &lt;a href=&#34;https://gist.github.com/dutchiechris/98ba718278cca3c1b7ecbc546aad22f0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;modify.py&lt;/a&gt; and save in &lt;code&gt;/vm-storage/coreos/manage.py&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;nb&#34;&gt;cd&lt;/span&gt; /vm-storage/coreos
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;wget https://gist.githubusercontent.com/dutchiechris/98ba718278cca3c1b7ecbc546aad22f0/raw/a9b3707fd1fc6054b0e23ae308b473c0bec08fd1/manage.py
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;chmod &lt;span class=&#34;m&#34;&gt;755&lt;/span&gt; manage.py&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Review the script usage:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-text&#34; data-lang=&#34;text&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;[root@3-kvm1 coreos]# ./manage.py -h
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;usage: manage.py [-h] --prefix PREFIX --start START --count COUNT --action
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;                 ACTION [--ip IP]
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Manage some container host VMs.
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;optional arguments:
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  -h, --help       show this help message and exit
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  --prefix PREFIX  Prefix hostname for managed VMs
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  --start START    Suffix number for managed VMs
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  --count COUNT    Count of VMs to manage
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  --action ACTION  Action to take [create|start|stop|undefine]
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  --ip IP          Starting IP address if creating VMs or keyword &amp;#34;dns&amp;#34;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;Explanation:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--prefix PREFIX&lt;/code&gt;: sets the basename of the VM; i.e. lab-docker or lab&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--start START&lt;/code&gt;: sets the machine number to start with; i.e. 1 or 5.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--count COUNT&lt;/code&gt;: sets the number of VMs to be managed incrementing from &lt;code&gt;--start&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--action ACTION&lt;/code&gt;: &lt;code&gt;create&lt;/code&gt; to create and customize a new VM copy from the master coreos and start it, &lt;code&gt;start&lt;/code&gt; to start an already defined VM, &lt;code&gt;stop&lt;/code&gt; to stop an already defined VM, and &lt;code&gt;undefine&lt;/code&gt; to stop and undefine existing VM&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--ip IP&lt;/code&gt;: Sets the IP address to start from and will increment for each VM.  For example if set to &lt;code&gt;10.64.28.150&lt;/code&gt; and count is 5 then the 1st created would be .150 and 2nd would be .151, etc.  If set to &lt;code&gt;dns&lt;/code&gt; a DNS lookup for the hostname will occur and the IP will be used.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run the script to create 2 VMs:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/manage.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Check if the VMs are running and login the console of one of them:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/virsh-console.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Some helpful libvirt provided CLIs to run on your KVM host:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh list --all &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;show all defined VMs&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh console coreos1 &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;connect to console&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh start coreos1 &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;startup&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh shutdown coreos1 &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;nice shutdown&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh destroy coreos1 &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;hard power off&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;virsh undefine coreos1 &lt;span class=&#34;o&#34;&gt;(&lt;/span&gt;remove config&lt;span class=&#34;o&#34;&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Check if Docker is working on your new container host VM by SSHing a &lt;code&gt;docker run&lt;/code&gt; command to your new container hosts:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/docker-run.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;blockquote&gt;
  &lt;p&gt;Booyah!  Pretty sweet, huh?&lt;/p&gt;

&lt;/blockquote&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;For the GUI equivalent experience go in your web browser to https://INSERT-KVM-IP-HERE:8001 and login as earlier.  Now click the Virtualization tab and you should see something like this:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/kimchi-vms.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Click the options menu and choose to view console:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/kimchi-vms-options.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;View the new browser tab and login using your credentials:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/kimchi-vms-vnc.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;When you are all done you can shutdown/undefine them just as quickly as you made them:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-12-30/manage-stop.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 data-hextra-search-id=&#34;summary&#34;&gt;Summary&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;summary&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#summary&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;After using these steps you have a container lab where in seconds you can create/destroy many container hosts so you can spend your time exploring the new tech [and not the infrastructure underneath].&lt;/p&gt;
&lt;p&gt;In future blogs I will use this infra to explore various tech extending the list below:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://www.beginswithdata.com/2017/01/02/docker-swarm-on-container-lab/&#34;&gt;Docker Swarm on Container Lab&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As always, comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Announcing NetApp Harvest 1.3</title>
      <link>https://www.beginswithdata.com/2016/11/14/harvest-1-3-released/</link>
      <pubDate>Mon, 14 Nov 2016 08:31:03 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2016/11/14/harvest-1-3-released/</guid>
      <description>
        
        
        &lt;p&gt;I am pleased to announce that &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Harvest v1.3&lt;/a&gt; is now available on the NetApp Support Toolchest! This feature release includes lots of new counters,  dashboard panels, and of course, bug fixes.&lt;/p&gt;
&lt;h2 data-hextra-search-id=&#34;major-new-features-in-version-13&#34;&gt;Major new features in version 1.3:&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;major-new-features-in-version-13&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#major-new-features-in-version-13&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;Support for ONTAP 9.0 and 9.1&lt;/li&gt;
&lt;li&gt;Support for OnCommand Unified Manager 6.4, 7.0, 7.1&lt;/li&gt;
&lt;li&gt;Updated dashboards that leverage Grafana 3.x features&lt;/li&gt;
&lt;li&gt;New counters and dashboard panels:
&lt;ul&gt;
&lt;li&gt;MetroCluster (FCVI)&lt;/li&gt;
&lt;li&gt;Backend disk/tape adapters&lt;/li&gt;
&lt;li&gt;NFS v4 and NFS v4.1&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Logfile rotation&lt;/li&gt;
&lt;li&gt;Ability to set multiple templates which are merged to make running template (ex: &lt;code&gt;template = default,my-extra-counters.conf&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Native installer packages for Debian/Ubuntu (.deb) and RHEL/CentOS (.rpm)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I updated many dashboards with new panel types in Grafana 3.x including tables and gauges and am pleased with how they turned out!&lt;/p&gt;
&lt;p&gt;Some examples:
&lt;img src=&#34;https://www.beginswithdata.com/post/2016-11-14/cluster.png&#34; alt=&#34;New cluster dashboard&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-11-14/node.png&#34; alt=&#34;New node dashboard&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;h4 data-hextra-search-id=&#34;more-views&#34;&gt;&lt;em&gt;&lt;strong&gt;More views&lt;/strong&gt;&lt;/em&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;more-views&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#more-views&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;The MetroCluster dashboard should be really useful to those who want to measure and analyze these configs.  The NFSv4 and v4.1 stats will give customers a view into performance of those protocols that was missing before.  And maybe most important, especially with the faster release cycles from NetApp engineering I&amp;rsquo;ve updated Harvest to work with the latest ONTAP and OnCommand Unified Manager versions.&lt;/p&gt;
&lt;h4 data-hextra-search-id=&#34;easier-install&#34;&gt;&lt;em&gt;&lt;strong&gt;Easier install&lt;/strong&gt;&lt;/em&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;easier-install&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#easier-install&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;Installing and upgrading has also gotten easier and Harvest is now available using native .deb and .rpm packages in addition to the previous .tgz source package.  For those with v1.2 you can simply install the native package on top of your original source install, import the newest dashboards, and you are upgraded to v1.3!&lt;/p&gt;
&lt;h4 data-hextra-search-id=&#34;all-in-one-virtual-appliance&#34;&gt;&lt;em&gt;&lt;strong&gt;All-in-one virtual appliance&lt;/strong&gt;&lt;/em&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;all-in-one-virtual-appliance&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#all-in-one-virtual-appliance&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;Lastly, for those who want an all-in-one solution with a focus on ease of use I also invite you to check out Yann Bizeul&amp;rsquo;s &lt;a href=&#34;https://nabox.tynsoe.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NAbox&lt;/a&gt;.  In recent months this OVA template  has been updated with an admin web UI.  The NAbox (previously known as the ADVA) provides you the latest Graphite and Grafana software pre-installed, and web form simplicity to configure Harvest pollers.  No Linux skills required.  Highly recommended!&lt;/p&gt;
&lt;h4 data-hextra-search-id=&#34;support-and-download-information&#34;&gt;&lt;em&gt;&lt;strong&gt;Support and download information&lt;/strong&gt;&lt;/em&gt;&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;support-and-download-information&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#support-and-download-information&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;Download NetApp Harvest: &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;NetApp Harvest is supported via NetApp Communities.  Create a new discussion on the &lt;a href=&#34;http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Discussions/bd-p/oncommand-storage-management-software-discussions&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Communities site&lt;/a&gt; for any questions or support.  Please use a tag of “netapp-harvest” to make it easily identifiable.&lt;/p&gt;
&lt;p&gt;Stay tuned for future blogs where I go into more details on these latest features.&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>PiBakery to the rescue!</title>
      <link>https://www.beginswithdata.com/2016/11/09/pibakery-to-the-rescue/</link>
      <pubDate>Wed, 09 Nov 2016 21:41:08 +0100</pubDate>
      
      <guid>https://www.beginswithdata.com/2016/11/09/pibakery-to-the-rescue/</guid>
      <description>
        
        
        &lt;p&gt;Together with my colleague Clemens Siebler we were preparing for an upcoming &lt;a href=&#34;https://github.com/NetAppEMEA/hackathon-vol1-insight&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hacker&amp;rsquo;s night&lt;/a&gt; night we&amp;rsquo;ll run at &lt;a href=&#34;http://www.netapp-insight.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Insight Berlin&lt;/a&gt;.  We have a challenge for participants that includes a few different bits of tech, including some Raspberry Pis.  I volunteered to handle the Pis; hardware and software is my task. Prototyping on one Pi I got everything working as wanted.  But for the event, one Pi isn&amp;rsquo;t enough.&lt;/p&gt;
&lt;blockquote&gt;
  &lt;p&gt;&lt;em&gt;Okay, 30 Raspberry PIs.  Headless.  On Wifi.&lt;/em&gt;&lt;/p&gt;

&lt;/blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;Big sigh&lt;/strong&gt;&lt;/em&gt;, what next? Ideally, I&amp;rsquo;d like the out-of-boxing experience to be for the hackers, but we can&amp;rsquo;t have them transferring (and waiting for) hundreds of MBs each over conference WIFI.&lt;/p&gt;
&lt;p&gt;Enter scene: &lt;a href=&#34;http://www.pibakery.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PiBakery&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-11-09/pibakery-0.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;PiBakery is a cool little program that allows you to configure your Raspbian image from your PC (cross-platform - it&amp;rsquo;s written in Electron/node.js) and then flash it onto the sd card.  It has a drag and drop interface and you pick &amp;lsquo;on first boot&amp;rsquo; and &amp;lsquo;on every boot&amp;rsquo;  as your start-up option and then chain actions below like setting WIFI details, passwords, installing packages, etc.  You can export/import and the file is XML so you can tweak it raw too if you want.&lt;/p&gt;
&lt;p&gt;For me just raspbian + wifi + fixed IP would be enough but after clicking everywhere I learned there is no &amp;ldquo;set fixed IP&amp;rdquo; function!  Oh drat!  I could however pull a file from the network or run any program.  Well, lets check how you set a fixed IP on Raspbian Jessie.  It seems the file &lt;code&gt;/etc/dhcpcd.conf&lt;/code&gt; just needs a few lines appended to it with the details and &lt;a href=&#34;https://en.wikipedia.org/wiki/Bob%27s_your_uncle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bob&amp;rsquo;s your uncle&lt;/a&gt;.  Appending a few lines, hmmm, I could do that by simply chaining a few &amp;ldquo;run command&amp;rdquo; functions like &lt;code&gt;echo &amp;quot;something important&amp;quot; &amp;gt;&amp;gt; /etc/dhcpdc.conf&lt;/code&gt;.  A quick test showed it worked!&lt;/p&gt;
&lt;p&gt;For anyone looking to set a fixed IP with PiBakery just do this:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2016-11-09/pibakery-1.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Actual commands:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;pre&gt;&lt;code&gt;echo &amp;#34;&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf
echo &amp;#34;#Set fixed IP on WLAN&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf
echo &amp;#34;interface wlan0&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf
echo &amp;#34;static ip_address=192.168.6.21/24&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf
echo &amp;#34;static routers=192.168.6.1&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf
echo &amp;#34;static domain_name_servers=192.168.6.1&amp;#34; &amp;gt;&amp;gt; /etc/dhcpcd.conf&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;I also tried to restart networking but couldn&amp;rsquo;t get that to work, so just added a reboot step which of course did.&lt;/p&gt;
&lt;p&gt;To load all 30x Pis I just had swap the physical sd card, update the ip address, click Write, wait 2-3 min, and repeat.  And when the participants assemble their Pi and power it up it will join the network with a fixed IP to let the more interesting hacking begin :-)&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>You&#39;re invited....NetApp Insight 2016 Session scheduling is open!</title>
      <link>https://www.beginswithdata.com/2016/09/09/insight-2016/</link>
      <pubDate>Fri, 09 Sep 2016 19:47:52 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2016/09/09/insight-2016/</guid>
      <description>
        
        
        &lt;p&gt;This year at &lt;a href=&#34;http://www.netapp-insight.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Insight&lt;/a&gt; I&amp;rsquo;ll be presenting three sessions and want to pack the house so please join me!&lt;/p&gt;
&lt;hr&gt;
&lt;h4 data-hextra-search-id=&#34;62051-2--open-source-monitoring-using-harvest-graphite-and-grafana&#34;&gt;62051-2 - Open Source Monitoring Using Harvest, Graphite and Grafana&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;62051-2--open-source-monitoring-using-harvest-graphite-and-grafana&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#62051-2--open-source-monitoring-using-harvest-graphite-and-grafana&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;This one is brand new, and I’ll cover the solution, custom counter collection templates and more.  So if interested in Harvest/Graphite/Grafana don’t miss it!&lt;/p&gt;
&lt;p&gt;Schedule it: [&lt;a href=&#34;https://netappinsight2016lasvegas.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=62051&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Las Vegas&lt;/a&gt;, &lt;a href=&#34;https://netappinsight2016berlin.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=62051&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Berlin&lt;/a&gt;]&lt;/p&gt;
&lt;hr&gt;
&lt;h4 data-hextra-search-id=&#34;59851-2---monitoring-performance-using-netapp-oncommand-performance-manager-and-api-extensions&#34;&gt;59851-2 - Monitoring Performance Using NetApp OnCommand Performance Manager and API Extensions&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;59851-2---monitoring-performance-using-netapp-oncommand-performance-manager-and-api-extensions&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#59851-2---monitoring-performance-using-netapp-oncommand-performance-manager-and-api-extensions&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;I’m lending a hand to Dhiman (primary presenter) because I’ve delved into the APIs a bit with Harvest and can share my experiences.  If you want to get perf data out of ONTAP systems be here to learn your options.&lt;/p&gt;
&lt;p&gt;Schedule it: [&lt;a href=&#34;https://netappinsight2016lasvegas.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=59851&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Las Vegas&lt;/a&gt; &lt;a href=&#34;https://netappinsight2016berlin.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=59851&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Berlin&lt;/a&gt;]&lt;/p&gt;
&lt;hr&gt;
&lt;h4 data-hextra-search-id=&#34;62053-3-tt---advanced-performance-analysis&#34;&gt;62053-3-TT - Advanced Performance Analysis&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;62053-3-tt---advanced-performance-analysis&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#62053-3-tt---advanced-performance-analysis&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h4&gt;&lt;p&gt;Last year was a hit, so I’ve reworked it with 4 new examples, discussion about ONTAP 9 perf enhancements, and extended to 2 hrs for Q/A.  If you want to get your perf on sign up!  &lt;em&gt;&lt;strong&gt;Note: Tech Team (NetApp and partners) only; sorry&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Schedule it: [&lt;a href=&#34;https://netappinsight2016lasvegas.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=62053&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Las Vegas&lt;/a&gt; &lt;a href=&#34;https://netappinsight2016berlin.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=62053&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Berlin&lt;/a&gt;]&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;I&amp;rsquo;m really looking forward to Insight so if you&amp;rsquo;ll be attending be sure to find me at the event, or hit me up on Twitter to make sure we meet!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Optimizing Azure ExpressRoute with NetApp Private Storage</title>
      <link>https://www.beginswithdata.com/2015/12/22/optimizing-azure-expressroute-netapp-private-storage/</link>
      <pubDate>Tue, 22 Dec 2015 21:49:05 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/12/22/optimizing-azure-expressroute-netapp-private-storage/</guid>
      <description>
        
        
        &lt;p&gt;Not too long ago &lt;a href=&#34;http://www.netapp.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp&lt;/a&gt;, &lt;a href=&#34;http://www.microsoft.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Microsoft&lt;/a&gt;, and &lt;a href=&#34;http://www.interxion.com/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Interxion&lt;/a&gt; came together to sponsor a hybrid cloud lab with &lt;a href=&#34;http://www.netapponcloud.com/netapp-private-storage&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Private storage for Azure&lt;/a&gt; nearby my office in Amsterdam.  When someone was needed to set this up I jumped at the chance!&lt;/p&gt;
&lt;p&gt;For those unaware of the solution, the basic premise is to use the Hyperscaler providers for elastic compute while maintaining ownership of the data and gaining the rich data management features of NetApp.  So use cases like adopting a &amp;lsquo;cloud first&amp;rsquo; strategy, DR in the cloud, dev/test in the cloud, cloud bursting, cloud hopping, and more can be experienced by customers and prospects first-hand in this lab. Contact me if interested to participate!&lt;/p&gt;
&lt;p&gt;Anyway, this post isn&amp;rsquo;t about the &amp;lsquo;why&amp;rsquo; but rather the &amp;lsquo;how&amp;rsquo;, and more specifically, the &amp;lsquo;how&amp;rsquo; of getting the most out of the Azure ExpressRoute service.&lt;/p&gt;
&lt;p&gt;After the hardware was shipped we had a short onboarding at Interxion (awesome service by the way; thanks Hans!), installed a switchless clustered Data ONTAP system, a couple of Cisco servers, and a Cisco Nexus switch in our cabinet. Next, Interxion pulled a pair of fibers through and hooked us up to their &lt;a href=&#34;http://www.interxion.com/sectors/enterprise/private-cloud-connection/cloud-connect/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Cloud Connect&lt;/a&gt; platform, which is our gateway to Azure (and more providers in the future).  With the physical tasks done there is no need to visit the data center again.&lt;/p&gt;
&lt;p&gt;Next we setup networking within Azure, and provisioned an ExpressRoute (ER) to our cabinet.  The steps are in NetApp Tech Report &lt;a href=&#34;http://www.netapp.com/us/media/tr-4316.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;TR-4316&lt;/a&gt;.  The guide has a few small mistakes or outdated info (cloud moves fast!) but nothing that couldn&amp;rsquo;t be solved by a actually reading error messages and a bit of Google.  After setting up an SVM with IP addresses on the Azure network it didn&amp;rsquo;t take long to access iSCSI, SMB, and NFS storage from Azure VMs.&lt;/p&gt;
&lt;p&gt;Then the fun started.  To start we had a 200Mbit ER and we could max that out pretty easily.  We boosted the speed to 1000Mbit (you can modify an existing circuit up but if you want to go down you have to destroy and create a new one so beware!) and were expecting to be able saturate the link.  Expecting&amp;hellip;but we just couldn’t do it.&lt;/p&gt;
&lt;h3 data-hextra-search-id=&#34;how-to-optimize-for-azure-expressroute-throughput&#34;&gt;How to optimize for Azure ExpressRoute throughput&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;how-to-optimize-for-azure-expressroute-throughput&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#how-to-optimize-for-azure-expressroute-throughput&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Choose a VM with lots of resources.  If you haven&amp;rsquo;t worked with a hyperscaler before it might come as a surprise that even though you have a 10Gbit vNIC it doesn&amp;rsquo;t mean you have 10Gbit worth of network access.  Far from it actually.  I was unable to find any official listing or SLA for bandwidth provided, but this old &lt;a href=&#34;https://social.msdn.microsoft.com/forums/azure/en-US/775a7d55-4ad5-438c-be2b-1f10e2eaa0d6/network-capacity-within-cluster-datacenter-and-across-datacenter&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;table&lt;/a&gt; seems to be reasonably correct for A1-A3 as confirmed by some basic testing. I actually saw a bit more with the A4 VM (8 cores, 14GB RAM) coming in at about 2Gbit.   So moral of the story, if you need bandwidth go for a VM with lots of resources like an A4 or equivalent in the DS line.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Remember the &lt;a href=&#34;https://en.wikipedia.org/wiki/Bandwidth-delay_product&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bandwidth Delay Product&lt;/a&gt;. Although ExpressRoute is low latency (&amp;lt;2ms in our case) it is not LAN latency (50-200us is typical).   For this reason we need to insure the TCP window size is large enough to not limit us on a per session basis. For iSCSI, Clustered Data ONTAP has a default window size of 131400 bytes. If you &lt;a href=&#34;http://www.speedguide.net/bdp.php&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;calc the BDP&lt;/a&gt; with this latency and window size it maxes out at about 64MB/s.  To go higher we can adjust this on the NetApp up to 262800 bytes at the SVM level using:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-text&#34; data-lang=&#34;text&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cdot::&amp;gt; set advanced
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cdot::*&amp;gt; iscsi modify -vserver -tcp-window-size 262800&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;p&gt;The value will take effect for new connections, so either stop/start the iscsi service, down/up each iscsi lif, or reboot your hosts. Check your connections have the correct TCP window size using&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-text&#34; data-lang=&#34;text&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;cdot::&amp;gt; iscsi connection show&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Remember &lt;a href=&#34;https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;TCP congestion avoidance&lt;/a&gt;. If packet loss occurs (ex: some intermediate device decides to police our traffic) then TCP congestion avoidance kicks in and starts sending data at a lower rate.  Over time the rate increases until packet loss occurs and again the data rate is reduced to avoid congestion. To cope with packet loss you want at least a handful of TCP sessions active because each one one implements its own TCP congestion avoidance allowing you to achieve higher total throughput and get as close as possible to the promised bandwidth. When using iSCSI simply make additional iSCSI sessions from the client.  You can use the same source/dest IP if you like, although if you want to get both links in the ER active you want at least 2 unique IP address pairs. In my testing I needed 4 to 12 sessions, with MPIO set to Least Queue Depth policy, to approach the promised bandwidth from a single host.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Configure BGP to balance traffic over both links.  ExpressRoute includes two links for redundancy and traffic from Azure will be balanced over both links. For traffic going to Azure this is determined by the BGP configuration made on your own switch, which by default uses a single path. To enable multiple paths on a Cisco Nexus switch add the &amp;lsquo;maximum-paths&amp;rsquo; setting like this:&lt;/p&gt;
&lt;div class=&#34;hextra-code-block hx:relative hx:mt-6 hx:first:mt-0 hx:group/code&#34;&gt;

&lt;div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-text&#34; data-lang=&#34;text&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;router bgp 64512
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  vrf vrf-100
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;    address-family ipv4 unicast
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;      network 192.168.100.0/24
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;        maximum-paths 2&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&#34;hextra-code-copy-btn-container hx:opacity-0 hx:transition hx:group-hover/code:opacity-100 hx:flex hx:gap-1 hx:absolute hx:m-[11px] hx:right-0 hx:top-0&#34;&gt;
  &lt;button
    class=&#34;hextra-code-copy-btn hx:group/copybtn hx:cursor-pointer hx:transition-all hx:active:opacity-50 hx:bg-primary-700/5 hx:border hx:border-black/5 hx:text-gray-600 hx:hover:text-gray-900 hx:rounded-md hx:p-1.5 hx:dark:bg-primary-300/10 hx:dark:border-white/10 hx:dark:text-gray-400 hx:dark:hover:text-gray-50&#34;
    title=&#34;Copy code&#34;
    aria-label=&#34;Copy code&#34;
    data-copied-label=&#34;Copied!&#34;
  &gt;
    &lt;div class=&#34;hextra-copy-icon hx:group-[.copied]/copybtn:hidden hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
&lt;div class=&#34;hextra-success-icon hx:hidden hx:group-[.copied]/copybtn:block hx:pointer-events-none hx:h-4 hx:w-4&#34;&gt;&lt;/div&gt;
  &lt;/button&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Choose an appropriate &lt;a href=&#34;https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpngateways/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vNET ExpressRoute Gateway&lt;/a&gt;; SKUs are available with throughput ranging from 500-2000Mbps. You are allowed to have 4 vNET Gateways per vNET, but each must be connected to an ExpressRoute in a different location.  So in practice it means for a given vNET there is a 1:1 relationship between vNET gateways and NPS locations.  It also means that for a given vNET your maximum throughput is limited to 2000Mbps.  I got this info from a colleague and have to say that in my testing I didn&amp;rsquo;t specify a SKU and saw around 2000Mbps which is more than expected from the table. So something to be aware of if testing.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 data-hextra-search-id=&#34;what-throughput-to-expect&#34;&gt;What throughput to expect&lt;span class=&#34;hx:absolute hx:-mt-20&#34; id=&#34;what-throughput-to-expect&#34;&gt;&lt;/span&gt;
    &lt;a href=&#34;#what-throughput-to-expect&#34; class=&#34;subheading-anchor&#34; aria-label=&#34;Permalink for this section&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;With the above setup and a 1Gbit ER I was able to achieve about 90MB/s reads + 90MB/s writes from a single host, and 200MB/s when writing from two hosts.  Unfortunately at the time I had the 1Gbit ER up I didn&amp;rsquo;t know of the BGP maximum-paths setting yet, and I downgraded my link to save $$, so I wasn&amp;rsquo;t able to verify max read throughput with both both links active. With my current 200Mbit ER though I can achieve 40MB/s reads (2 x 200Mbit), so I assume that with a 1Gbit 200MB/s reads would be possible.&lt;/p&gt;
&lt;p&gt;Still, with the smallest 200Mbit ER and two hosts generating workload I can fill both directions pretty well :-)
&lt;img src=&#34;https://www.beginswithdata.com/post/2015-12-22/nps-200mbit.png&#34; alt=&#34;nps-200mbit&#34;  loading=&#34;lazy&#34; /&gt;
&lt;img src=&#34;https://www.beginswithdata.com/post/2015-12-22/nps-200mbit-grafana.png&#34; alt=&#34;nps-200mbit-grafana&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Awesome performance scaling with NetApp Clustered DATA ONTAP</title>
      <link>https://www.beginswithdata.com/2015/11/20/awesome-performance-scaling-netapp-clustered-data-ontap/</link>
      <pubDate>Fri, 20 Nov 2015 12:54:11 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/11/20/awesome-performance-scaling-netapp-clustered-data-ontap/</guid>
      <description>
        
        
        &lt;p&gt;I just got back from NetApp Insight 2015 in Berlin and what a time it was!  One highlight for me was seeing the power of NetApp Clustered Data ONTAP scaling a database workload from 22k IOPs on a Hybrid array all the way up to 1.2 Million IOPs on an all flash one with no disruption, while at the same time maintaining hundreds of snapshots and clones. &lt;strong&gt;Wow&lt;/strong&gt;. What other storage system on the planet can do that? None.&lt;/p&gt;
&lt;p&gt;And if that wasn&amp;rsquo;t enough, I got goosebumps when I saw the near real-time stats coming in from &lt;a href=&#34;https://www.beginswithdata.com/2015/09/16/netapp-advanced-performance-monitoring-with-harvest-graphite-and-grafana/&#34;&gt;Harvest&lt;/a&gt; &amp;amp; &lt;a href=&#34;http://netofrombrazil.com/2015/12/02/graphora-docker-image-oracle-real-time-performance-statistics/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Graphora&lt;/a&gt; being displayed in a beautiful Grafana dashboard.&lt;/p&gt;
&lt;p&gt;The demo is up on YouTube so if you couldn&amp;rsquo;t attend live check out the 6 minute snippet that made me proud:&lt;/p&gt;
&lt;p&gt;&lt;span class=&#34;embed-youtube&#34; style=&#34;text-align:center; display: block;&#34;&gt;&lt;iframe class=&#39;youtube-player&#39; type=&#39;text/html&#39; width=&#39;640&#39; height=&#39;390&#39; src=&#39;https://www.youtube.com/embed/O94ac9uWo24?version=3&amp;#038;rel=1&amp;#038;fs=1&amp;#038;autohide=2&amp;#038;showsearch=0&amp;#038;showinfo=1&amp;#038;iv_load_policy=1&amp;#038;start=4445&amp;#038;end=4878&amp;#038;wmode=transparent&#39; allowfullscreen=&#39;true&#39; style=&#39;border:0;&#39;&gt;&lt;/iframe&gt;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;Neto and Jeff did many more demos, so if you liked that rewind the video to 55m 29s to see some NetApp cloud scenarios possible TODAY using a MS SQL database and:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Replicating it from on premise to near the cloud with &lt;a href=&#34;http://www.netapp.com/us/solutions/cloud/private-storage-cloud/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Private Storage (NPS)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Using that replicated storage to spin up a bunch of dev/test instances in AWS in a few seconds using no extra storage&lt;/li&gt;
&lt;li&gt;Clustering the SQL Server with nodes in both AWS and Azure and then surviving a cloud fail&lt;/li&gt;
&lt;li&gt;Creating a Cloud ONTAP system in AWS natively on AWS EBS Storage&lt;/li&gt;
&lt;li&gt;Backing the database up to an AltaVault with StorageGrid object store using S3, and then moving those backups to AWS S3&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Awesome stuff.  Awesome time at Insight.  I can&amp;rsquo;t want until next year!&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Grafana 2.5 support with NetApp Harvest</title>
      <link>https://www.beginswithdata.com/2015/11/11/grafana-2-5-support-with-netapp-harvest/</link>
      <pubDate>Wed, 11 Nov 2015 19:02:47 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/11/11/grafana-2-5-support-with-netapp-harvest/</guid>
      <description>
        
        
        &lt;div class=&#34;hx:overflow-x-auto hx:mt-6 hx:flex hx:rounded-lg hx:border hx:py-2 hx:ltr:pr-4 hx:rtl:pl-4 hx:contrast-more:border-current hx:contrast-more:dark:border-current hx:border-blue-200 hx:bg-blue-100 hx:text-blue-900 hx:dark:border-blue-200/30 hx:dark:bg-blue-900/30 hx:dark:text-blue-200&#34;&gt;
  &lt;div class=&#34;hx:ltr:pl-3 hx:ltr:pr-2 hx:rtl:pr-3 hx:rtl:pl-2&#34;&gt;&lt;svg height=1.2em class=&#34;hx:inline-block hx:align-middle&#34; xmlns=&#34;http://www.w3.org/2000/svg&#34; fill=&#34;none&#34; viewBox=&#34;0 0 24 24&#34; stroke-width=&#34;2&#34; stroke=&#34;currentColor&#34; aria-hidden=&#34;true&#34;&gt;&lt;path stroke-linecap=&#34;round&#34; stroke-linejoin=&#34;round&#34; d=&#34;M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z&#34;/&gt;&lt;/svg&gt;&lt;/div&gt;

  &lt;div class=&#34;hx:w-full hx:min-w-0 hx:leading-7&#34;&gt;
    &lt;div class=&#34;hx:mt-6 hx:leading-7 hx:first:mt-0&#34;&gt;&lt;strong&gt;UPDATE (11-Nov-2016):&lt;/strong&gt; Grafana 3.1.1 support is available with NetApp Harvest v1.3. Download NetApp Harvest v1.3 from the &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Support Toolchest&lt;/a&gt;. If you must use Harvest 1.2.2 and desire dashboard updates keep reading this blog entry.&lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;Recently Grafana released v2.5 with a host of new features:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;http://grafana.org/blog/2015/10/28/Grafana-2-5-Released.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-11-11/grafana_2_5_graphic.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Included in this major release was some additional error checking on singlestat dashboards to ensure only one metric series was passed; previously multiple were allowed and it would just display the first one (see &lt;a href=&#34;https://github.com/grafana/grafana/issues/2745&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Issue 2745&lt;/a&gt; if interested).&lt;/p&gt;
&lt;p&gt;Anyway, some of my default Harvest dashboard panels were passing multiple metrics which resulted in a blank panel and tooltip &amp;ldquo;Multiple Series Error&amp;rdquo;:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-11-11/error.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;I went through all the dashboards and fixed all of these errors, and tweaked some other dashboards (some new panels, modified display on others, fixed some inconsistent colors) resulting in an updated set of default dashboards.  These will be included in an upcoming Harvest release (along with other features and fixes) but to deliver compatibility with Grafana v2.5 sooner I&amp;rsquo;d thought I&amp;rsquo;d post just these dashboards for you now.&lt;/p&gt;
&lt;p&gt;So if you want to run Grafana 2.5 with Harvest 1.2.2 here&amp;rsquo;s what you can do:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Install and configure NetApp Harvest 1.2.2 from the &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Toolchest&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Update your Grafana server to v2.5; see the &lt;a href=&#34;http://grafana.org/download/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Grafana download page&lt;/a&gt; for instructions&lt;/li&gt;
&lt;li&gt;Download this set of updated &lt;a href=&#34;https://www.beginswithdata.com/post/2015-11-11/netapp-harvest-grafana-2.5.zip&#34;&gt;dashboards&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;On the Harvest poller server extract them into &lt;code&gt;/opt/netapp-harvest/grafana&lt;/code&gt; (or &lt;code&gt;/opt/harvest/grafana&lt;/code&gt; if using the VA). That directory contains the 1.2.2 default dashboards so you can just overwrite them.  Make sure you don&amp;rsquo;t leave the zip file in that directory as in the next step it would try to import it too!&lt;/li&gt;
&lt;li&gt;Run &lt;code&gt;/opt/netapp-harvest/netapp-manager -import&lt;/code&gt; (or &lt;code&gt;/opt/harvest/netapp-manager -import&lt;/code&gt; if using the VA) to load them into Grafana. This command will overwrite all existing dashboards with matching names, so if you edited the default dashboards previously and didn&amp;rsquo;t do a &amp;lsquo;save as&amp;rsquo; make sure you do that before importing!&lt;/li&gt;
&lt;li&gt;Do a browser F5 refresh and enjoy the goodness!&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>NetApp metrics collector for Graphite:  OPM vs Harvest</title>
      <link>https://www.beginswithdata.com/2015/10/09/netapp-metrics-collector-for-graphite-opm-vs-harvest/</link>
      <pubDate>Fri, 09 Oct 2015 10:37:58 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/10/09/netapp-metrics-collector-for-graphite-opm-vs-harvest/</guid>
      <description>
        
        
        &lt;p&gt;So you have Graphite + Grafana up and running and want to get metrics in there from your NetApp storage systems and discovered there are two options:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://library.netapp.com/ecmdocs/ECMP12406791/html/GUID-E7845BEB-6006-4CAA-BA5C-DADA8F3D7F13.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;OnCommand Performance Manager &amp;lsquo;External Data Provider&amp;rsquo; feature&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Harvest collector on the NetApp ToolChest&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Your next question was probably which one is right for me?&lt;/p&gt;
&lt;p&gt;The answer is, as with many things, &amp;ldquo;it depends&amp;rdquo;.&lt;/p&gt;
&lt;p&gt;Some history here.  A long time ago (maybe 2yrs now) I created Harvest and socialized it internally.  The product team that owns OPM saw it, recognized the value of integrating with open source metrics platforms, and decided to create the &amp;ldquo;External Data Provider&amp;rdquo; feature which they shipped in the OPM 1.1 release. Meanwhile, Harvest continued being used internally and at some customers with continual feedback/enhancement loops occurring. Harvest then went &amp;ldquo;public&amp;rdquo; in September on the ToolChest. So that&amp;rsquo;s how we got here and might help you understand why there are two options and give context to some of the differences.&lt;/p&gt;
&lt;p&gt;To compare the two I created this table:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-10-09/OPM__vs__Harvest.png&#34; alt=&#34;OPM vs Harvest as data source&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s step through each one:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Polling interval&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; has a fixed polling interval of every 5 minutes from the clusters. It uses this interval for its own DB and UI and the metrics sent to Graphite share this interval.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; collects every 1 minute by default. The time is however configured per poller, and any time (lower or higher) can be set; the lowest you can go is largely dependent on the API response time from the cluster. I don&amp;rsquo;t think there are a lot of use cases to go lower than 1 min (data overload to view and to store!) but it&amp;rsquo;s possible. For large clusters it&amp;rsquo;s also quite possible that Harvest won&amp;rsquo;t be able to keep up at 1 min intervals and you will have to either reduce the objects collected or poll less frequently.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Counter inventory&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; chose simplicity here and has three presets to chose from with a varying level of details. It does not allow further customization or selection of your own counters. Today it has about 115 counters in the most detailed preset.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; went for flexibility and uses a config template to control the counters collected. The default one is quite comprehensive with more than 500 counters and is the result of my field experience at NetApp and lots of feedback from others. You can also create your own templates to add or remove counters based on your need. Over time I expect to add more, for example NFSv4 is on my to-do list.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Dashboards provided&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; doesn&amp;rsquo;t have any. Hopefully someone will make some and post them to the NetApp Communities!&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; does include dashboards, a bunch of them. Maybe so many it can be a bit overwhelming! I plan to create some YouTube videos of stepping through them so you can better understand what you&amp;rsquo;re looking at; another item on my to-do list.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Data ONTAP 7-mode&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; is built from the ground up for cDOT and has no support for 7-Mode.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; was built for both cDOT and 7-Mode and supports both just fine.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Storage capacity information&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; doesn&amp;rsquo;t do storage capacity, and since there is no &amp;ldquo;External Data Provider&amp;rdquo; in Unified Manager (OCUM) you can&amp;rsquo;t get capacity info from it either.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; has a poller type for OCUM 6.x and can fetch capacity info at many levels (aggr, vol, lun, qtree, aggr, etc) allowing cDOT dashboards that have a blend of performance and capacity information. I didn&amp;rsquo;t add a poller type for UM 5.2 (and don&amp;rsquo;t plan to) so you don&amp;rsquo;t have capacity info for 7-mode systems; sorry.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Scaling&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; does a lot of analytic processing and has some documented maximums for clusters it can monitor. With cDOT 8.3 it also uses a collection method from the cluster that is more efficient (not individual API calls but fetching an archive file that the cluster has prepared for it) making it lighter weight for the monitored cluster.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; doesn&amp;rsquo;t do any analytic processing but rather calculates a the metric value, normalizes it, and sends it off to Graphite. As such there are no explicit maximums of a cluster it can monitor. It uses API calls to gather all data (even on cDOT 8.3) and for large clusters that might mean collection can&amp;rsquo;t complete in 1 min and you have to adjust frequency or collected objects as mentioned earlier. But, you do have a tool that can collect metrics from any cluster size or configuration.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Ease of use&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; has a menu driven interface and all monitored clusters will automatically send metrics to Graphite. And since it&amp;rsquo;s part of OPM there is nothing extra to install or maintain.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; is driven by a config file which needs to be edited manually; installation of a new cluster means editing that config file. It also has to be installed separately and maintained over time.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Downloadable from&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt; is an official product and is available on the NetApp Support Site.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; is a ToolChest tool and is available on the NetApp Support ToolChest site.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Supported by&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;OPM&lt;/strong&gt;, including the &amp;ldquo;External Data Provider&amp;rdquo; feature, is supported by NetApp Support and you can open a support case if something doesn&amp;rsquo;t work or you need help.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Harvest&lt;/strong&gt; is a ToolChest tool and is not supported by NetApp Support. If something doesn&amp;rsquo;t work you can either fix it yourself (Harvest is written in Perl and delivered as code) or you can post your question on the &lt;a href=&#34;http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Discussions/bd-p/oncommand-storage-management-software-discussions&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Communities&lt;/a&gt; site.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;So the choice of OPM vs Harvest is really a matter of requirements and preference. Both have pros/cons but hopefully with the info above you will be able to pick the right solution for your environment.&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>CPU Utilization Monitoring of many nodes</title>
      <link>https://www.beginswithdata.com/2015/09/25/cpu-utilization-monitoring-of-many-nodes/</link>
      <pubDate>Fri, 25 Sep 2015 09:00:39 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/09/25/cpu-utilization-monitoring-of-many-nodes/</guid>
      <description>
        
        
        &lt;p&gt;I got an email from a colleague asking if I had any ideas for a customer need of presenting CPU busy across a set of systems.  He wanted both average and percentile values over a specified timeframe.  In his case it was NetApp cDOT storage array nodes, but the concept could be applied to whatever group of systems you happen to be interested in.&lt;/p&gt;
&lt;p&gt;I hadn&amp;rsquo;t yet used the repeat function added in &lt;a href=&#34;http://docs.grafana.org/guides/whats-new-in-v2-1/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Grafana 2.1&lt;/a&gt; and since I was interested in a single metric across many systems it seemed a natural fit.  I used the panel repeat function based on the node template item.  So for every node you get a panel.&lt;/p&gt;
&lt;p&gt;And here&amp;rsquo;s the result across all my lab nodes with singlestat:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.beginswithdata.com/post/2015-09-25/cpu_highlights.png&#34;&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-09-25/cpu_highlights.png&#34; alt=&#34;cpu_highlights&#34;  loading=&#34;lazy&#34; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;And then the drilldown (on a subset for the screenshot) with graph panels:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.beginswithdata.com/post/2015-09-25/cpu_drilldown.png&#34;&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-09-25/cpu_drilldown.png&#34; alt=&#34;cpu_drilldown&#34;  loading=&#34;lazy&#34; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;So with this I&amp;rsquo;ve got a dynamic dashboard for at a glance singlestat panels and more detailed graph panels to drilldown if needed.  Nice!&lt;/p&gt;
&lt;p&gt;If you use &lt;a href=&#34;https://www.beginswithdata.com/2015/09/16/netapp-advanced-performance-monitoring-with-harvest-graphite-and-grafana/&#34;&gt;NetApp Harvest&lt;/a&gt; grab this dashboard and give it a try: &lt;a href=&#34;https://www.beginswithdata.com/post/2015-09-25/db_netapp-detail-cpu-singlestats.zip&#34;&gt;db_netapp-detail-cpu-singlestats&lt;/a&gt;.  After downloading and unpacking, import the json to Grafana  (import on the dashboard list) and you should be all set.  Or, use that as your starting point and customize further!&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>NetApp Advanced Performance Monitoring with Harvest, Graphite, and Grafana</title>
      <link>https://www.beginswithdata.com/2015/09/16/netapp-advanced-performance-monitoring-with-harvest-graphite-and-grafana/</link>
      <pubDate>Wed, 16 Sep 2015 19:26:50 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/09/16/netapp-advanced-performance-monitoring-with-harvest-graphite-and-grafana/</guid>
      <description>
        
        
        &lt;p&gt;I&amp;rsquo;m pleased to announce that NetApp Harvest is now posted to the &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp ToolChest&lt;/a&gt;! NetApp Harvest is a data collector that fetches a wide set of performance metrics from NetApp cDOT and 7-mode storage systems, and capacity info from Unified Manager 6+, and posts it to an open-source &lt;a href=&#34;http://graphiteapp.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Graphite&lt;/a&gt; metrics server.  It also imports 19 dashboards containing over 500 graphs into &lt;a href=&#34;http://grafana.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Grafana&lt;/a&gt; to make viewing it all a snap.&lt;/p&gt;
&lt;p&gt;I am just &lt;strong&gt;thrilled&lt;/strong&gt; with the results and I hope you are too.  Rather than try to explain it in words I created a 10 minute video that gives a quick tour and some steps to get you started:&lt;/p&gt;
&lt;iframe src=&#34;https://www.youtube.com/embed/wmaiFXNN7GE&#34; width=&#34;560&#34; height=&#34;315&#34; frameborder=&#34;0&#34; allowfullscreen=&#34;allowfullscreen&#34;&gt;&lt;/iframe&gt;
&lt;p&gt;Want it?  Great!  All is available without any cost or license key so get to it!&lt;/p&gt;
&lt;p&gt;Here are the steps from the video:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Install Graphite and Grafana&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Articles-and-Resources/How-to-install-Graphite-and-Grafana/ta-p/109456&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Instructions for Ubuntu, RHEL 6 &amp;amp; 7 (offline and internet connected)&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Install NetApp Harvest&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Instructions and software here on the NetApp ToolChest&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enjoy your Dashboards and share!&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Questions, Experiences, Tips, or your custom dashboards visit the NetApp Communites section &lt;a href=&#34;http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Discussions/bd-p/oncommand-storage-management-software-discussions&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;OnCommand Storage Management Software Discussions&lt;/a&gt; and tag your post with “netapp-harvest”&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;This won&amp;rsquo;t be the last post about NetApp Harvest, Graphite, and Grafana, so stay tuned and let the metrics fun begin!&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Installing Graphite and Grafana on RHEL 6, 7, or Ubuntu in under 30 minutes</title>
      <link>https://www.beginswithdata.com/2015/09/14/installing-graphite-and-grafana-on-rhel-6-7-or-ubuntu-in-under-30-minutes/</link>
      <pubDate>Mon, 14 Sep 2015 14:31:19 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/09/14/installing-graphite-and-grafana-on-rhel-6-7-or-ubuntu-in-under-30-minutes/</guid>
      <description>
        
        
        &lt;p&gt;&lt;img src=&#34;https://www.beginswithdata.com/post/2015-09-14/graphite-grafana.png&#34; alt=&#34;&#34;  loading=&#34;lazy&#34; /&gt;&lt;/p&gt;
&lt;p&gt;The open source metrics software Graphite is &lt;strong&gt;&lt;em&gt;brilliant&lt;/em&gt;&lt;/strong&gt;.  To get it up and running however is &lt;strong&gt;&lt;em&gt;not so&lt;/em&gt;&lt;/strong&gt;.  &lt;/p&gt;
&lt;p&gt;The ‘official’ documentation is quite generic, and the packaged versions (on EPEL and via PIP) have their own issues on some Linux flavors.  So I usually end up searching Google to hopefully find a working recipe for the Linux flavor of the day on someone’s blog.  And when I’ve done that invariably it still doesn’t work.  A bit more googling and hacking and some minutes or hours later depending on my luck then ta-da, &lt;strong&gt;SUCCESS&lt;/strong&gt;!&lt;/p&gt;
&lt;p&gt;Could I do it again with the minimum steps possible?  Not a chance.  Then ask me to do it without internet access (so passing install files via a jumphost) and it turns into a multi-hour or multi-day adventure.&lt;/p&gt;
&lt;p&gt;Faced with this problem, and a desire to get Graphite and Grafana up and running quickly, I wrote an installation document for the following common scenarios:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Red Hat Enterprise Linux (RHEL) 6.6 (internet connected or via jumphost)&lt;/li&gt;
&lt;li&gt;Red Hat Enterprise Linux (RHEL) 7.1  (internet connected or via jumphost)&lt;/li&gt;
&lt;li&gt;Ubuntu 14.04 (internet connected)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;So you can pick a totally free option (Ubuntu), or go with an enterprise option that will hopefully fit your corporate standards (RHEL).  And if you have no internet access (often the case from a management network) we also have steps for an offline install on RHEL.&lt;/p&gt;
&lt;p&gt;With these steps you can be up and running in &lt;strong&gt;under 30 min&lt;/strong&gt; with minimal linux skills!&lt;/p&gt;
&lt;p&gt;Once you’re up, if you’re a NetApp shop you can configure metrics from the &lt;a href=&#34;https://library.netapp.com/ecmdocs/ECMP1552962/html/GUID-E7845BEB-6006-4CAA-BA5C-DADA8F3D7F13.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;OnCommand Performance Manager (OPM) external data provider feature&lt;/a&gt;, or the &lt;a href=&#34;http://mysupport.netapp.com/tools/info/ECMLP2314554I.html?productID=61924&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NetApp Harvest collector found on the NetApp ToolChest&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Or if you’re not a NetApp shop, be happy that you have your metrics platform up and running and can start monitoring with something like &lt;a href=&#34;https://collectd.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;collectd.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The instructions are downloadable here, no login required:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Articles-and-Resources/How-to-install-Graphite-and-Grafana/ta-p/109456&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Quick Start: Installing Graphite and Grafana&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
    <item>
      <title>Volume move; as impressive as ever</title>
      <link>https://www.beginswithdata.com/2015/09/13/volume-move-as-impressive-as-ever/</link>
      <pubDate>Sun, 13 Sep 2015 20:46:57 +0000</pubDate>
      
      <guid>https://www.beginswithdata.com/2015/09/13/volume-move-as-impressive-as-ever/</guid>
      <description>
        
        
        &lt;iframe src=&#34;https://www.youtube.com/embed/KlzYXks0rcY&#34; width=&#34;560&#34; height=&#34;315&#34; frameborder=&#34;0&#34; allowfullscreen=&#34;allowfullscreen&#34;&gt;&lt;/iframe&gt;
&lt;p&gt;Back in 2012 when NetApp first added SAN protocol support to clustered Data ONTAP I created a short video to show how easy and non-disruptive it is to move a volume (containing one or more LUNs) around in a cluster.  Although a bit dated, the feature is as powerful as ever!&lt;/p&gt;
&lt;p&gt;From an operational perspective being able to move data without impacting users or reconfiguring clients is huge. If you want to re-balance for capacity or performance (including from spinning disk to hybrid to all-flash!), or replace hardware during a tech refresh, you can just do it!&lt;/p&gt;
&lt;p&gt;It’s like VMware’s storage vMotion on steriods:  Any storage protocol, any client type, and your dedupe and compression and snapshot and replication and all the rest all keep working too.  Just one more feature in Clustered Data ONTAP!&lt;/p&gt;
&lt;p&gt;Comments are always welcome!&lt;/p&gt;

      </description>
    </item>
    
  </channel>
</rss>
