Cloud

When setting up storage for your Google Kubernetes Engine (GKE) environment, customers can choose between block storage (Persistent Disk, or PD) or file storage (Filestore Enterprise). While both storage types are fully supported with containers on GKE including operations such as migrating containers across hosts for use cases such as upgrades or failover, Filestore Enterprise requires very little expertise on behalf of the customer to operate, especially during a failover

At Storage Spotlight, we introduced Filestore Enterprise multishare for use with GKE Autopilot and Standard clusters, and today, that feature is generally available. In this blog post, we look at considerations for choosing block vs. file in a GKE environment, and how to use the new Filestore Enterprise multishare feature. 

Block vs. file storage considerations

When you choose block-based Persistent Disk, you get best-in-class price/performance with extensive selection of multiple PD types, making it popular with many GKE customers. However, with PD, customers need to have expertise in storage systems. When using PD, the file system logic is in the host. This coupling means that during migrations, the host must cleanly shut down the container, unmount the file system, reattach the PD to the target host, and mount the file system. Only then can it boot the container. While GKE manages a lot of these operations automatically, in the case of failover there are potential file system and disk corruption issues. Users will need to run some cleanup processes (“fsck”) on the mounted volume before it can be used. 

Contrast that with Filestore Enterprise, which provides a fully managed regional file system that is decoupled from the host, and which does not need you to perform any infrastructure operations to attach/detach volumes. In addition, you also benefit from storage that can be simultaneously read to and written by multiple (hundreds to thousands) containers. That’s why Filestore Enterprise is popular across a variety of use cases: multiple developers accessing data e.g. JupyterHub; content management systems such as WordPress, Drupal, IBM FileNet; interactive SaaS applications e.g., project or process management tools; and cases where multiple users read and write and analytics applications that consume multiple files e.g., genomic processing.

Optimizing storage utilization with Filestore Enterprise multishare 

The introduction of Filestore Enterprise multishare for use with your GKE Autopilot and Standard clusters makes using Filestore even more compelling. Filestore has been available to use with the Container Storage Interface (CSI) driver with a 1TB starting capacity. Customers wanted to bin pack volumes on Filestore instances like they do with containers on GKE nodes, and were using directories to drive better utilization of Filestore instances. However, this solution is not seamless and lacks enterprise features such as capacity isolation, observability per volume, and support for CMEK. Filestore multishare support for GKE brings the best of both worlds: the granular resource sizing that Kubernetes customers are used to along with the dependability of Filestore Enterprise. Just like how containers are bin-packed on a node for better efficiency, multiple persistent volumes can be packed on a Filestore Enterprise instance to increase storage utilization and decrease costs. Additionally, Filestore multishare saves network resources by sharing network elements such as IP addresses across multiple shares and thus enhancing the application scalability. 

You can provision Persistent Volumes (PVs) ranging from 100 GiB up to 1TiB on a Filestore instance. GKE seamlessly manages all related resources. Further, as shown in Figure 1, when you request storage via the kubernetes Persistent Volume Claim (PVC), underneath the hood, the GKE Filestore CSI driver seamlessly bin-packs volume requests on Filestore Enterprise instances and also creates new Filestore instances once space on a particular instance runs out. As the volumes are deleted, the GKE Filestore CSI driver seamlessly shrinks or deletes the underlying Filestore instances.

“Filestore Enterprise gives us a regional storage system that is decoupled from compute and works well with containers on GKE. Now with Multishare, we can optimize our usage to drive better cost efficiency,” said Ajay Kemparaj, Mandiant. “This feature has now tipped our decision to make Filestore Enterprise our default storage solution on GKE.”

Deployment and configuration considerations

If you decide to use the new Filestore multishare capability, here are a few things to keep in mind:

Noisy neighbors: Enabling shares allows you to reuse space of the underlying Filestore instance to save on costs. However, it also means that you share performance resources such as IOPS and throughput. 

Trust boundary: Since all storage requests (via PVCs) that use Filestore multishare can be packed on the same underlying Filestore instance, we recommend that PVCs share the same trust boundary. If you need stronger isolation (e.g., you are requesting storage from two different customers and want to keep the data isolated), we recommend using Filestore single-share instances (where each PV maps to a unique filestore instance), which is also supported on GKE with CSI dynamic provisioning. 

Regional availability: Filestore Enterprise is a regional service and delivers 99.99% availability SLA. Consequently, you can deploy GKE clusters across three zones within a region and all clusters have access to the same Filestore NFS share. This deployment architecture provides protection from a zonal outage, as all Filestore Enterprise data is synchronously replicated across three zones.

Getting started

Filestore Enterprise multishare is provisioned using standard Container Storage Interface semantics including dynamic provisioning. To see a detailed example on how two Kubernetes deployments with persistent volumes share an underlying Filestore Enterprise instance see the user guide.

What a year! Over here at Google Cloud, we’re winding things down, but not before taking some time to reflect on everything that happened over the past twelve months. 

Inspired by the custom Spotify Wrapped playlist playing in our earbuds, we pulled the data about the best-read Google Cloud news posts of the year, to better understand which stories resonated most with you. 

Many of your favorite stories came as no surprise, as they tracked with major news, product launches, and events. But there were some sleeper hits in there too — stories whose viral success and staying power took us a bit by surprise. We also uncovered some fascinating data about the older posts that you keep coming back to, month after month, year after year (stay tuned for more on that in 2023). So, without further ado, here are the top 22 Google Cloud news stories of 2022, according to you, our readers.

Here’s what to know about changes to kubectl authentication coming in GKE v1.26

How Google Cloud blocked the largest Layer 7 DDoS attack at 46 million rps

Protecting customers against cryptomining threats with VM Threat Detection in Security Command Center

Introducing the next evolution of Looker, your unified business intelligence platform

Even more pi in the sky: Calculating 100 trillion digits of pi on Google Cloud

Introducing AlloyDB for PostgreSQL: Free yourself from expensive, legacy databases

Introducing Blockchain Node Engine: fully managed node-hosting for Web3 development

Introducing Google Public Sector

Google + Mandiant: Transforming Security Operations and Incident Response

Raising the bar in Security Operations: Google Acquires Siemplify

The L’Oréal Beauty Tech Data Platform – A data story of terabytes and serverless

Build a data mesh on Google Cloud with Dataplex, now generally available

Google Cloud launches new dedicated Digital Assets Team

Contact Center AI reimagines the customer experience through full end-to-end platform

Unveiling the 2021 Google Cloud Partner of the Year Award Winners

Automate Public Certificates Lifecycle Management via RFC 8555 (ACME)

AlloyDB for PostgreSQL under the hood: Intelligent, database-aware storage

Bringing together the best of both sides of BI with Looker and Data Studio

Supercharge your event-driven architecture with new Cloud Functions (2nd gen)

Announcing the 2022 Accelerate State of DevOps Report: A deep dive into security

Making Cobalt Strike harder for threat actors to abuse 

Securing tomorrow today: Why Google now protects its internal communications from quantum threats

Recognize any of your favorites? We thought you might. See anything you missed? Now’s your chance to catch up.

Let’s take a deeper look at these top posts as they landed throughout the year. 

January

Raising the bar in Security Operations: Google Acquires Siemplify (#10)

We set off some new year’s fireworks by acquiring security operations specialist Siemplify, combining their proven security orchestration, automation and response technology with our Chronicle security analytics to build a next-generation security operations workflow.

Google Cloud launches new dedicated Digital Assets Team (#13)

News flash: blockchain technology has huge potential. So it was no big surprise that readers responded with gusto to the news of Google Cloud’s new Digital Assets Team, whose charter is to support customers’ needs in building, transacting, storing value, and deploying new products on blockchain-based platforms.

February

Protecting customers against cryptomining threats with VM Threat Detection in Security Command Center (#3)

Who wants their VMs to be hijacked by hackers mining crypto? No one. To help, we added a new layer of threat detection to our Security Command Center that can help detect threats such as cryptomining malware inside virtual machines running on Google Cloud. 

Here’s what to know about changes to kubectl authentication coming in GKE v1.26 (#1)

The open-source Kubernetes community made a big move when it decided to require that all provider-specific code that currently exists in the OSS code base be removed (starting with v1.26). We responded with a blockbuster post (the #1 post of the year, in terms of readership) that outlines how this move impacts the client side. 

Supercharge your event-driven architecture with new Cloud Functions (2nd gen) (#19)

Developers eyeing serverless platforms responded with enthusiasm to news of our next-generation Functions-as-a-Service product, which offers more powerful infrastructure, advanced control over performance and scalability, more control around the functions runtime, and support for triggers from over 90 event sources. 

Build a data mesh on Google Cloud with Dataplex, now generally available (#12)

Building a data mesh is hard to do. But doing so lets data teams centrally manage, monitor, and govern their data across all manner of data lakes, data warehouses, and data marts, so they can make the data available to various analytics and data science tools. With Dataplex, data teams got a new way to do just that.

March

The L’Oréal Beauty Tech Data Platform – A data story of terabytes and serverless (#11)

Serverless, event-driven architecture, cross-cloud analytics… This customer story from L’Oréal about how it built its Beauty Tech Data Platform had it all. 

Contact Center AI reimagines the customer experience through full end-to-end platform(#14)

Customers rely on contact centers for help when they encounter urgent problems with a product or service, but contact centers often struggle to provide timely help. To bridge this gap with the power of AI, Google Cloud built Contact Center AI (CCAI) to streamline and shorten this time to value. CCAI Platform, the addition announced here, expanded this effort by introducing end-to-end call center capabilities.

Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) (#16)

With this announcement, Google Cloud customers were able to acquire public certificates for their workloads that terminate TLS directly or for their cross-cloud and on-premises workloads using the Automatic Certificate Management Environment (ACME) protocol. This is the same standard used by Certificate Authorities to enable automatic lifecycle management of TLS certificates.

April

Bringing together the best of both sides of BI with Looker and Data Studio (#18)

When Google Cloud acquired Looker in 2020 for its business intelligence and analytics platform, inquiring minds instantly began asking what would become of Data Studio, Google’s existing self-serve BI solution. This blog began to answer that question.

May 

Introducing AlloyDB for PostgreSQL: Free yourself from expensive, legacy databases (#6)

Live from Shoreline at Google I/O, we made one of our largest product announcements of the year, launching a PostgreSQL database that can handle both transactional and analytical workloads, without sacrificing performance.

AlloyDB for PostgreSQL under the hood: Intelligent, database-aware storage (#17)

Readers couldn’t get enough about AlloyDB, piling on to learn about the inner workings of its database-aware storage (not to mention its columnar engine). 

June/July 

Even more pi in the sky: Calculating 100 trillion digits of pi on Google Cloud (#5)

A follow up to a reader favorite from 2019, we broke the record (again) by calculating the most digits of pi, leaning into significant advancements in Google Cloud compute, networking and storage. 

Unveiling the 2021 Google Cloud Partner of the Year Award Winners (#15)

Who consistently demonstrates a creative spirit, collaborative drive, and a customer-first approach? Google Cloud partners, of course! With this blog, we were proud to recognize you and to call you our partners!

Introducing Google Public Sector (#8)

The U.S. government had been asking for more choice in cloud vendors who could support its missions, and protect the health, safety, and security of its citizens. With the announcement of Google Public Sector, a subsidiary of Google LLC that will bring Google Cloud and Google Workspace technologies to U.S. public sector customers, we delivered.

August

How Google Cloud blocked the largest Layer 7 DDoS attack at 46 million rps (#2)

Distributed denial-of-service (DDoS) attacks have been increasing in frequency and growing in size exponentially. In this post, we described how Cloud Armor protected one Google Cloud customer from the largest DDoS attack ever recorded — an attack so large that it was like receiving all of the requests that Wikipedia receives in a day in just 10 seconds. 

September

Google + Mandiant: Transforming Security Operations and Incident Response (#9)

Here, we took a moment to reflect on the completion of our acquisition of threat intelligence firm Mandiant. Bringing Mandiant into the Google Cloud fold will allow us to deliver a security operations suite to help enterprises globally stay protected at every stage of the security lifecycle, and focus on eliminating entire classes of threats. 

Announcing the 2022 Accelerate State of DevOps Report: A deep dive into security (#20)

For eight years now, DevOps professionals have pored over the results of DORA’s annual Accelerate State of DevOps Report. This year’s installment focused on the relationship between security and DevOps, using the Supply-chain Levels for Secure Artifacts (SLSA) and NIST Secure Software Development frameworks. 

October

Introducing the next evolution of Looker, your unified business intelligence platform (#4)

In April, we began to lay out our strategy for Looker and Data Studio. At Google Cloud Next ‘22, we took the next step, consolidating the two under the Looker brand umbrella, and adding important new capabilities. 

Introducing Blockchain Node Engine: fully managed node-hosting for Web3 development (#7)

Remember how in January we said that blockchain has a lot of potential? About that. News of the fully managed Blockchain Node Engine node-hosting service took readers by storm, catapulting it to the top ten of 2022, with just over two months left in the year. 

November/December

Making Cobalt Strike harder for threat actors to abuse (#21)

Legitimate versions of Cobalt Strike are a very popular red team software tool, but older, cracked versions are often used by malicious hackers to spread malware. We made available to the security community a set of open-source YARA Rules that can be deployed to help stop the illicit use of Cobalt Strike. 

Securing tomorrow today: Why Google now protects its internal communications from quantum threats (#22)

Google and Google Cloud have taken steps to harden our cryptographic algorithms used to protect internal communications against quantum computing threats. We explain here why we did it, and what challenges we face to achieve this type of future-proofing.

That’s a wrap!

Barring any last minute surprises, we’re pretty confident that what we have here is the definitive list of your favorite news stories of 2022 — you’ve got great taste. We can’t wait to see what stories inspire you in the new year. Happy holidays, and thanks for reading!

Editor’s note: Yariv Adan has been involved in the creation of some of Google’s most well-known and astonishing products—helping make complex Artificial Intelligence (AI) accessible and useful to billions of people. For a long time, he’s been living with chronic pain. A naturally lively and curious person, his personal journey has taught him resilience and compassion.

How long have you been working in Artificial Intelligence? What have you built?

I’ve been working in AI off and on for 25 years, after first becoming interested in college. Twenty years ago, I used relatively simple AI models to spot anomalies in business operations. I joined Google in 2007 to work on other things and got back up to speed on AI in 2013. I have been fortunate to be part of the team that developed Google Assistant, Google Lens and Google Duplex, which creates a natural conversation with computers. At the beginning of 2022, I came over to Google Cloud to work on conversational AI for enterprises.

How different is it to build AI for businesses, versus consumers?

Working with Google Assistant and Duplex technology convinced me that society is headed for revolutionary changes in how people and computers interact. For now though, there’s more value and need in business, which is likely because problems and challenges are better scoped and constrained. For example, Contact Center AI helps provide answers and solutions to problems without having to wait for a human agent—this means lower costs and better customer satisfaction. The benefit is clear and measurable. Another example is content moderation in online gaming, which keeps out bad behavior.  

That’s an impressive body of work, setting that aside, what else has been going on?

In 2014, I had a motorcycle accident that left my right arm basically unusable. As a result of the accident, some of my nerves were crushed and ever since then, my brain has been registering chronic pain. If you’ve ever had a tough dental procedure, or a sleepless night with a migraine, it’s a similar feeling, except it has been going on constantly for years. The pain is always there—when I try to sleep, when I’m working, even when I’m traveling and having fun with my wife and four kids.

How could that be?

The brain generates the feeling of pain as a defense and safety mechanism. If you break your leg, pain keeps you from trying to walk on it because that would make the injury worse. When you have an infection, pain is why you pay attention and treat it. In my case, the system that is responsible for generating and controlling pain is broken.

I tried pain medication, but it was unhelpful, and caused brain fog, memory issues, slow metabolism, and many other side effects that wrecked my performance at work. Besides, the pain finds its way back! I quit taking medication in 2017, and learned other ways to live with the pain. For many years I just worked on things I was excited about and focused less on career progression.

What has the experience shown you?

There’s been a lot of personal growth, especially in understanding that pain and other disabilities are not the opposite of happiness. Don’t get me wrong, chronic pain is often annoying and frustrating; I’ve learned, though, that how much I let it control my life and mood is up to me.

Even more, I’ve learned how many other people face similar challenges. Individuals often start work after a sleepless night, or go hours while the condition worsens. “Simple” tasks like typing notes or following a presentation become hellish. They avoid necessary medications because they worry side effects will impact their work. Worst of all, they feel they must hide what they’re going through from managers and peers. 

It’s heartbreaking, enraging, and crazy. According to the World Health Organization, about 15% of the world’s population has some kind of disability. People cycle in and out of disability too, meaning that a great number of us will have one at some point in life.   

What can be done? 

Equity across hiring, performance management, promotions and retention, and accommodations for the disabled are well behind similar diversity, equity, and inclusion initiatives for other groups. The vast and varied range of disabilities, the complex and fragmented regulatory landscape, and people’s reluctance to discuss their condition all make it difficult. 

Solving it will take a long time, but everyone can start making a difference today. By educating yourself, you increase awareness, empathy, and helpful behavior in the workplace. That makes the life of your disabled colleagues easier. We must bring this topic out of the shadows and address it without discomfort or shame. There are successful people with disabilities all around us. As hard as mine is, I strongly believe that it is a badge of honor.

Startups worldwide turn to Google Cloud tools to build fast on a strong and easy to use platform that helps them get to market and launch products faster, all while building on the cleanest cloud in the industry. Startups leverage Google Cloud and our Google for Startups Cloud Program to go from idea to IPO, and there are a variety of products on Google Cloud that can help them.

Here are the 8 top products that startups use on Google Cloud to innovate and grow:

1. Firebase for app development

Speed up innovation with Firebase, a mobile development platform that’s fully integrated with Google Cloud. Work in a simpler cloud environment, easily pull in products or services, and build your apps faster.

2. Cloud SQL for database needs

Build your startup’s foundation with Cloud SQL, a fully managed relational database solution that integrates with Google Cloud services. Create and connect to your first database in minutes and scale with a single API call.

3. AI and machine learning products

Solve tough problems with AI and machine learning products, built with the best of Google’s technology. Train deep learning and machine learning models cost-effectively so you can iterate and innovate faster.

4. BigQuery for data analytics

Drive agility with BigQuery, a serverless, cost-effective, multi-cloud data warehouse. Query streaming data in real time, predict business outcomes with built-in machine learning, and share analytics with just a few clicks.

5. Google Kubernetes Engine (GKE) for containers

Unlock faster, more secure app development with GKE, the most scalable Kubernetes platform. Streamline operations with release channels that fit your business needs and leave cluster monitoring to Google engineers.

6. Looker for data visualization

Get more from your data to keep moving ahead of the competition with Looker, a trusted business intelligence and data platform. Generate real-time reports and get insights at the right time with proactive alerts.

7. Cloud Run for serverless computing

Create scalable containerized apps in any programming language on Cloud Run, a fully managed compute platform. Pair it with container tools like Cloud Build and Docker, and only pay when your code is running.

8. Cloud Armor for security

Protect your startup from Web attacks with Cloud Armor, a leading Distributed Denial-of-Service (DDOS) defense service. Use it with an HTTP Load Balancer for Managed Instance Groups across regions to keep your workloads highly available and secure.

To learn more about products best-suited to the unique demands of startups, check out our startups solution page. Our team is looking forward to discussing how these products can help you. If you’re not already in the program, you can get started here. 

If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, andsign up for our communications to get a look at our community activities, digital events, special offers, and more.

Security operations teams are facing a “perfect storm” of challenges from nation-state actors turning their attention to financial crime, to rising uncertainty and potential complexities because of rapidly advancing cloud migration and IoT adoption, to the long-lamented skills shortage. 

Now imagine having to face this trifecta without visibility into your IT and security infrastructure because scale and cost concerns have impeded your ability to ingest and monitor critical telemetry. To help organizations rethink threat detection and broader security team efforts, we showcased at this month’s Google Cloud Security Talks a keynote on why DEI is a cybersecurity imperative, Mandiant’s review of lessons for organizations two years after SolarWinds, how Google Chrome already helps protect your business (and what more you can make do with it,) how best to encourage your organization to pursue Zero Trust, and three key ways in which Chronicle Security Operations can help your organization scale threat detection and actionable outcomes. These include:

1. Operationalizing threat intelligence

Harnessing the power of threat intelligence to acquire context, prioritize risk, and act on detections requires more than just ingesting a set of indicator of compromise (IOC) feeds. Threat intelligence must have breadth and depth of coverage to provide custom details on the tools, tactics, and procedures of threat actors. Explore why part of the “magic” behind Google Cloud’s security is the sheer scale of threat intelligence we can share with our customers. 

2. Driving detection as code

We want to enable organizations that are more advanced in their cybersecurity maturity to build their own custom detections and rules, which means offering our customers a powerful detection-authoring platform. Chronicle Security Operations can help analysts build specific detections around, for example, known bad files or a specific registry key change. We also demonstrated how to create more complex detections, such as flagging process execution patterns or crafting a suspicious-behavior trigger from a specific activity sequence.

3. Leveraging curated detections

With curated detections, which we announced in August, we are putting the power of Google Cloud’s intelligence in the hands of security operations teams everywhere. Our detections offer actionable, ready-to-use threat detection content curated, built, and maintained by Google Cloud Threat Intelligence (GCTI) researchers. These detection sets cover a wide variety of threats for your network and beyond, including attacks such as ransomware, remote-access tools (RAT), infostealers, data exfiltration, and suspicious activity.

You can watch all nine of our Security Talks recorded sessions here, and keep an eye out for our next Security Talks, coming in March 2023.

Editor’s note: This post is part of an ongoing series on IT predictions from Google Cloud experts. Check out the full list of our predictions on how IT will change in the coming years.

Prediction: By 2025, over half of all business applications will be built by users who do not identify as professional developers today

As organizations evolve, more and more development work is being taken on by teams and individuals outside of traditional IT departments. As this trend continues, we believe that by 2025, more than half of all business applications will be built by individuals who are not professional developers.

Think about it: How long do your teams have to wait for new features or apps because development and IT teams are overwhelmed? Backlogs for enterprise IT teams are notoriously long, with the average backlog running anywhere between 3 and 12 months. With low-code and no-code tools you don’t have to wait on another team’s backlog. Instead, business users can create applications and workflow automations on their own, with little-to-no programming skills required. 

These applications and tools will be built collaboratively with developers to provide the guardrails needed to keep the business secure, while still empowering non-technical users to create and deliver their own solutions. As a result, companies are able to free up dev team and IT bandwidth so they can deliver on their roadmap faster, or add feature requests. 

We’re not alone in this belief — numerous entities track the rise of low-code and no-code development, and predict it will be used for everything from new product initiatives, end-to-end process automation, sales and customer engagement, to customer service and support.

Organizations are already moving in this direction. Globe Telecom reduced targeted business process turnaround time by 80% thanks to digital experiences built by its citizen developers with Appsheet and Google Workspace, with new organizations exploring this approach every day. To learn more about how you can expand your organization’s bench of citizen developers, check out my talk from Google Cloud Next ‘22.