Cloud

Today’s workplace is getting more and more data intensive. This is why, even in a modern data stack world full of incredible business intelligence tools, “good ol’ reliable” spreadsheets are still commonly used to wrangle data problems big and small. 

Data is everywhere, so why aren’t insights?

While companies recognize the value data can provide for their business, tapping into it isn’t always easy. Spreadsheets are a fantastic tool for users to start turning raw data into something meaningful, which is why knowing how to use spreadsheets for data analysis remains critically important for users across industries — it’s the top use case cited by 63% of users when it comes to their sheets. And yet, more than half of users say they still struggle to make sense of their data in sheets. For every power user or data analyst who knows how to make the most of spreadsheets, there are countless business users, each with their own specific questions.  

The increasing demands on today’s knowledge workers, the new decision makers, are only exacerbating these challenges. Not only do you need the skills to analyze and act on data quickly, but you need live access to this data so you’re making decisions based on up-to-date information in the moment, not what was happening earlier today, yesterday, or last week. 

With the industry adopting cloud data warehouses and the rise of the data cloud, we are witnessing the rise of the connected sheet. Google Cloud BigQuery makes it easy for any information worker to tap into Google’s data cloud, leveraging the ease and use of a Connected Sheets experience. 

But let’s face it, spreadsheet users come in all shapes and sizes. Not everyone has the skills of an analytics engineer. You’ve got everyone from spreadsheet experts who can perform VLOOKUPS in the blink of an eye, to the humble beginner just mastering how to write a simple SUM() formula. The truth is spreadsheets are everywhere, but aren’t for everyone. Simply bringing them to the cloud isn’t going to fix that. 

As one of our ThoughtSpot customers said, “Sheets and Excel are pretty powerful, but not easy for everyone to use. It’s one thing to get the data into sheets, but once it’s there it’s difficult to drill, ask ad-hoc questions, or simply build charts and visualizations.”

Simply put: Spreadsheets can be hard. Creating a chart or data visualization from spreadsheets can be even more challenging. Drilling into spreadsheet data can be complicated, even for power users.

But it doesn’t have to be. We teamed up with Google Cloud to invent something new to bring the power of Connected Sheets to everyone. Introducing ThoughtSpot for Google Sheets.

Unleashing insights from data in sheets

ThoughtSpot for Sheets is a standalone native app plugin for Google Sheets that brings a true self-service analytics experience to your sheets data. With ThoughtSpot for Sheets, creating charts and visualizations is as simple as searching. All you need is some data in Sheets and a curious mind.

Let’s take a look.

Bringing data to Connected Sheets

There’s a plethora of ways to get data into Sheets. Many savvy users are turning to popular data integration tools like Supermetrics and Coefficient.io to connect live data from their business apps to Google Sheets for ad-hoc analysis. Other people like a quick and dirty copy and paste. However, with so much business data moving to data clouds, you can now query the trove of customer data in BigQuery through Sheets.

Getting started

ThoughtSpot for Sheets is simple to launch. Do a quick search on Google Workspace Marketplace or find it here. A simple click and you’ve installed the app. Depending on your  organization, you might have to send a brief friendly message to authorize the plugin from your Google Workspace Admin.

With ThoughtSpot installed, you’re ready to connect to some data — just go to the Data tab and select “Connect to BigQuery”. Now you can start to explore tables in your business data set or connect to some of the public data sets available from Google Cloud’s Analytics Hub.

Once your Sheet is connected, the fun really begins. 

No data modeling required. That’s right. No star schemas or snowflake schemas. Just insights. 

With one click, you’ll be able to leap over the biggest hurdle people face with data. Simply select ThoughtSpot from your Extensions menu, and the app handles the rest. Depending on the data size, in just a few moments the app automatically reads, understands, and categorizes your data into different types (DATES, MEASURES, ATTRIBUTES) all in a easy to use modal that sits right on top of your Sheet for easy exploration.

Create content with search

Once data is in Sheets, ThoughtSpot makes it easy to ask questions of your data. Using the search bar or selecting from the side panel, you can start to create new visuals automatically. The user experience was designed for the data explorer, and the data explorer in all of us just waiting to be unleashed. The search engine acts as a powerful compiler and gives you access to unlimited combinations of searches to uncover opportunities hidden in your Sheets. With search you can call up endless combinations of measures, attributes, and even filters in your data, so no stone is left unturned.

We even built in powerful keywords to take your analysis further, like “top” which generates the top n items from a sorted result. If your search has more than one measure, ThoughtSpot generates the top n items from the first measure in the search.

With search you can even travel through time with ease. The search engine lets you easily aggregate your data daily, weekly, monthly or get even more granular with your time slices like [last 2 weeks], [last 35 days], etc.

Drill anywhere

True self-service BI should be limitless, without making drilling into the data difficult. So we put the data search engine to work again to let you drill anywhere in your data with a single click. Say “so long” to VLOOKUPS. 

Don’t worry, you won’t get lost. We built in easy internal navigation that lets you easily undo/redo your query so you never lose track of that aha-moment.

Data storytelling with Slides & Sheets

For almost every user we surveyed, making sensible charts and visualizations from their spreadsheet data is a requirement, especially for data storytelling, yet more than half say doing so is challenging. In addition to being a valuable companion for Google Sheets, the app also integrates with Google Slides to make sharing, collaboration, and presenting data a breeze. When you are happy with the chart or visualization you created, you can Pin to Slides which then lets you create a new presentation or add to an existing deck.

Putting ThoughtSpot for Sheets to work

So in theory, this all sounds great. But it’s even more exciting when applied to the real world. 

Sales

Whether Salesforce, Gong, Outreach, Qualified, or Drift, there are so many data sources to help give sales professionals the edge.

With ThoughtSpot for Sheets, you have easy to use sales analytics to manage pipeline health, go deeper with prospecting, or make sure your next QBR with your customer or manager is grounded in actionable data.

Marketing

Google Sheets has become the workhorse of any marketer. Data from Google Ads, web analytics, HubSpot, and advertising spend all find their way into Sheets, begging for marketing analytics so users can understand customer and user trends, optimize programs, and uncover the next winning campaign.

Research

Universities all over the world leverage Google Workspace as their productivity suite of choice. Students and researchers now have access to a powerful self-service experience, conveniently right in the Google Sheets they already use.

Explore public datasets available from BigQuery and Analytics Hub from crypto and census data to geopolitical ad spend to boost data fluency and research projects.

Spreadsheets 2.0 and the dawn of the Connected Sheet is upon us. Time to get more with your new best friend ThoughtSpot. Install the app free, and let us know what you think.

Happy data exploring.

Data security is a key focus for organizations moving their data warehouses from on-premises to cloud-first systems, such as BigQuery. In addition to storage-level encryption, whether using Google-managed or customer-managed keys, BigQuery also provides column-level encryption. Using BigQuery’s SQL AEAD functions, organizations can enforce a more granular level of encryption to help protect sensitive customer data, such as government identity or credit card numbers, and help comply with security requirements.

While BigQuery provides column-level encryption in the cloud, many organizations operate in hybrid-cloud environments. To prevent a scenario where data needs to be decrypted and re-encrypted each time it moves between locations, Google Cloud offers a consistent and interoperable encryption mechanism. This enables deterministically-encrypted data (which maintains referential integrity) to be immediately joined with on-prem tables for anonymized analytics.

To achieve a BigQuery-compatible encryption on-prem, customers can use Tink, a Google-developed open-source cryptography library. BigQuery uses Tink to implement its SQL AEAD functions. We can use the Tink library directly to encrypt data on-prem in a way that can later be decrypted using BigQuery SQL in the cloud, and decrypt BigQuery’s column level-encrypted data outside of BigQuery. 

For our customers who want to use Tink with BigQuery, we have put together a few helpful Python utilities and samples in the BigQuery Tink Toolkit GitHub repo. Let’s first walk through an example of how to use Tink directly to encrypt or decrypt on-prem data using the same keyset used for BigQuery, followed by how the BigQuery Tink Toolkit can help simplify working with Tink. 

To start, we need to retrieve the Tink keyset. We’ll assume that KMS-wrapped keysets are being used. These keysets need to be stored in BigQuery to use with BigQuery SQL.  If needed, they can also be replicated to a secondary store on-prem.

Now that we have the encrypted keyset, we need to unwrap it to retrieve the usable Tink keyset. If Cloud KMS is not accessible from on-prem, the unwrapped keyset will need to be maintained in a secure keystore on-prem.

We can now use the keyset to generate a Tink primitive. This can be used to encrypt or decrypt data with the associated keyset. Note that different primitives should be used depending on whether the keyset is for a deterministic or nondeterministic key.

Once we have our cipher, we can use it to encrypt or decrypt data as needed.

We have provided the CipherManager class to help simplify this process, which handles four actions:

Retrieving the required keysets from a BigQuery table

Unwrapping those keysets

Creating a Tink cipher for each column

Providing a consistent interface to call encrypt and decrypt. 

We have also included a sample Spark job that shows how to use CipherManager to encrypt or decrypt columns for a given table. We hope these come in handy – happy Tinkering.

Editor’s note: Google Cloud supports a wide range of IPv6 capabilities. For an overview, check out this blog.

When people talk about the IPv4 Address exhaustion problem, it’s usually in the context of the  public IPv4 space. When IPv6 was introduced, the primary goal was to fix the address exhaustion problem by allocating internet-accessible addresses to every device. However, IP exhaustion issues in large enterprises are often related to the private RFC1918 address space. Large enterprises use the RFC1918 address space for their internal networks. And lately, technological advancements such as 5G, internet of things, mobile applications, connected devices, serverless and container-based services have expedited the exhaustion of the IPv4 private address space. 

One particularly interesting IPv6 capability you’ll find in Google Cloud is support for ULA (Unique Local Addresses), an IPv6 address space defined in RFC4193 that is analogous to the private IP space used in IPv4 defined in RFC 1918. 

Even though the IPv6 address space is gigantic compared to the IPv4 address space, enterprises require a private network that is not exposed to threats that you might find on the public internet. Like IPv4, IPv6 retains the notion of private addressing as a separate private address space that will not conflict with the public address space. IPv6 ULA addresses are routable within the scope of private networks, but not publicly routable on the global IPv6 internet, thus providing isolation for private workloads from the internet and other cloud customers. Further, you can allocate and use these addresses without arbitration by a central registration authority.

Google Cloud allows you to create internal IPv6 ULA subnets for private communication within a VPC. Any workloads with IPv6 ULA addresses allocated from these subnets are meant to be used for private communication within your network. Any workload that is meant to be publicly available can leverage external IPv6 with GUA (Globally Unique Addresses). Additionally, multi-nic VM instances may be dual-homed with both ULA (internal) and GUA (external) addresses.

Using ULA addressing for private communication in Google Cloud provides the following advantages over the traditional IPv4 RFC 1918 address space. 

1. VPCs with billions and billions of IPv6 addresses 

A /48 ULA range is allocated to each VPC and each subnet in the VPC will be allocated a /64 IPv6 address range from this /48 ULA range. Each VPC with a /48 ULA range can accommodate 65,536 /64 subnets. A dual-stack subnet is assigned both IPv4 and IPv6 address ranges. Each /64 subnet can accommodate 4 billion unique VM interfaces. Each IPv6-enabled VM will be assigned a /96 address range from the subnet, which provides you with 4 billion unique IPv6 addresses for each VM interface.

2.  Non-overlapping private IPv6 address space

When you create a ULA-enabled VPC, you are assigned a /48 range for your VPC, which can be used across all Google Cloud regions. You can use this aggregate range to simplify specification of ACLs, firewalls, and access controls in your on-prem or cross-cloud deployments.

Unlike RFC1918 addresses, which are meant to be reused across multiple networks, ULA addresses are meant to be unique. RFC4193 includes a description for a pseudo-random IP generator to help avoid overlaps. Google Cloud assigns all ULA addresses from the fd20::/20 range and ensures that each VPC network is assigned a unique /48 ULA prefix to avoid overlaps with other VPCs. This is beneficial when using VPC peering, which requires non-overlapping IPv6 address ranges for subnets in the peering VPCs. Ensuring uniqueness also eliminates the need for NAT to communicate between private networks. Google Cloud provides you the flexibility to choose a ULA range for your VPC that does not overlap with your on-prem/cross-cloud ULA ranges. 

Support for IPv6 ULA addressing opens up a world of possibilities for enterprises with large, complex systems, like those based on containers and microservices. To learn more about how to get started with IPv6 in Google Cloud, check out the documentation. We can’t wait for all the interesting things you will build using IPv6 ULA address space.

Transforming the customer experience of insurance claims processing is a key driver for transforming the insurance claims process itself. This needs a holistic approach to addressing different aspects of the claims processing workflow and modernizing the data that claims processing heavily relies on. Transforming claims processing can help reduce the manual touches from the process and lower processing time and costs, resulting in reduced claims’ payouts and higher policyholder satisfaction.

Studies by Verisk and Coalition Against Insurance Fraud show that auto insurers lose 14% of premiums a year due to claims leakage (about $29B). The annual cost of insurance fraud was as high as $60B and fraud occurs in about 10% of property-casualty insurance losses. Another study by PwC shows that 41% of customers are likely or more likely to switch providers due to a lack of digital capabilities. Inefficiencies in claims processing have a direct impact on the business and customer experience. 

We describe how Google Cloud technologies working in unison through implementing the defined reference architecture can help organizations transform their Insurance claims processing to reap measurable business benefits and enhance the policyholder experience. 

Introducing the Google Cloud insurance claims reference architecture 

Google Cloud’s reference architecture for transforming insurance claims processing is available through the Google Cloud architecture diagramming tool (you can find it under Reference Architectures – Financial Services – Insurance Claims). 

The reference architecture uses Google Cloud’s AI and data analytics platforms together with other Google Cloud technologies and supports the following use cases:

Claims submission / FNOL: Report claims online within minutes and enabling policyholders to upload claims documents at scale to the insurer’s app receiving the FNOL. 

Claims assignment and analysis: Classify submitted documents, analyze images, perform claims segmentation.

Claims decision, repair analysis: Self-service damage assessment, real-time claims status, automated or quick settlement.

Claims leakage analytics: Analyze and report on claims data to assess and identify potential and actual claims leak.

Reporting and insights: Fraud monitoring, real-time insights into claims workflows, use claims insights in marketing and sales.

How to use the reference architecture

The reference architecture provides a framework for using Google Cloud to transform the insurance claims process holistically. The architecture can be implemented as-is in its entirety or addressed one function at a time.

Every insurance provider is in a different stage of their claims transformation journey, with a differing appetite to risk and unique budgetary needs. That’s why the reference architecture can be adopted in one of the following modes – Tactical (start small with prioritized functions), Strategic (identify multiple functional concerns and address them together for a measurable impact), Transformational (go big).

What can you achieve with the reference architecture?

The Google Cloud insurance claims reference architecture provides the following features:

Claims submission

Consume FNOL submission via any channel – forms, paper PDFs, scanned images. 

Ingest the documents at scale for any form of claims data submission. 

Perform inline data transformation to store data for near and long term.

Claims analysis

Advanced analytics to intelligently accelerate claims processing. 

Parse form fields and tables from ingested documents.

Extract text from PDFs and images.

Use AI to classify claims documents, perform claims segmentation and run fraud detection models.

Asses damage in real time in claims scenarios requiring assessment of damage to property or vehicles. 

Claims decisions

Make a decision on claims either automatically or significantly faster than today.

Validate claims, perform adverse selection modeling scenarios.

Estimate the cost of repair in damage assessment scenarios.

Predict payments and automatically settle claims in case of STP (straight through processing).

In case of non-STP, provide all necessary intelligence to associates to make a quick decision.

Claims leakage analytics

Analyze and report on claims data to assess and identify potential and actual claims leak.

Create models and visualizations to gain insights into several parameters like average settlement cost trends, number of reserve changes per claim, average time of first contact of claimant, subrogation recovery trends and more such analytics.

Reporting and data leverage

Store and unify content at scale, so it’s available when needed where needed and to whoever needs it. 

Archive content for regulatory compliance at a fraction of the cost of primary storage. 

Unify select claims insights and share them via APIs with internal teams, such as marketing or sales for more personalized offerings.

Transforming the claims processing workflow can provide multiple benefits, for example: 

Faster multi-channel claims submission process and responsiveness can greatly increase customer satisfaction.

Data analytics and AI capabilities can create efficiencies that reduce the costs and increase the speed of claims adjustment.

Effective claims validation and management validation can improve the accuracy of claims’ payments.

Real-time insights can help cross-sell and up-sell insurance products based on policy holders’ needs.

Why transform insurance with Google Cloud

Further to the reference architecture we presented here, Google Cloud is uniquely positioned to transform an insurance business holistically. Here are a few technology enablers from Google Cloud that can aid in such a holistic transformation.

Google BigLake allows you to unify data from claims, underwriting and other insurance functions at scale, so it’s available to all lines of business including marketing and contact center.

Holistic data ecosystem that lets you build a datamesh to leverage data across various business lines like claims, underwriting and marketing with BigQuery for analytics, DataPlex to build a datamesh and Vertex AI for advanced AI and MLOps.

Google Earth Engine,  a geospatial processing service, helps perform geospatial processing at scale, powered by BigQuery for property risk exposure assessment and portfolio management. For claims adjusters and underwriters, it provides an interactive platform for geospatial driven analytics. 

Public datasets support claims analysts and actuaries for several scenarios including dynamic insurance pricing model using this dataset.

Solving big problems usually takes a combination of the right people and the right tools. SRE, DevOps, and IT operations teams in organizations both big and small have used Google Cloud’s built-in logging service, Cloud Logging, to troubleshoot faster, recognize trends easier, and scale operations more effectively. Additionally, customers have been building homegrown solutions that combine the powers of BigQuery and Cloud Logging to help them address operational and security challenges at massive scale. Last year we introduced Log Analytics, powered by BigQuery, so more customers can bring logs and advanced analysis together without having to build the connection themselves.  

Today, we are announcing the general availability of Cloud Logging’s Log Analytics (powered by BigQuery), a capability that allows you to search, aggregate and transform all log data types including application, network and audit log data at no additional cost for existing Cloud Logging customers. We are also launching three new Log Analytics capabilities. 

Multi-region support for US and EU region

Improved query experience to save and share queries

Support for custom retention up to 10 years

To get started, upgrade your existing log buckets to Log Analytics supported buckets. 

Same logs, same cost, more value with Log Analytics 

Log Analytics brings entirely new capabilities to search, aggregate, or transform logs at query time directly into Cloud Logging with a new user experience that’s optimized for analyzing logs data. 

Centralized logging –  By collecting and centrally storing the log data in a dedicated Log Bucket, it allows multiple stakeholders to manipulate their data from the same datasource. You don’t need to make duplicate copies of the data.

Reduced cost and complexity – Log Analytics allows reuse of data across the organization, effectively saving cost and reducing complexities. 

Ad hoc log analysis – It allows for ad-hoc query-time log analysis without requiring complex pre-processing. 

Scalable platform – Log Analytics can scale for observability using the serverless BQ platform and perform aggregation at petabyte scale efficiently

By leveraging BigQuery, Log Analytics breaks down data silos helping security, networking, developer and even business teams collaborate using a single copy of data. 

New features in this release

1. Multi-region support for Log Analytics buckets 

In addition to GA, we are also announcing multi-region support for Log Analytics with log buckets in the US and EU. These new multi-regions are available for log buckets that use Log Analytics and for those that don’t.  This means that you can now store and analyze your logs in the region that is most convenient for you, improving performance and reducing latency. 

2. Improved query experience

We are also improving the query experience by allowing users to save, share and re-use recent queries. This means that you can easily reuse and share your most important queries, saving time and making it easier to get the insights you need from your logs.

3. Retain logs up to 10 years in a Log Analytics bucket

We are rolling out the ability to support custom log retention. You can now store logs in the Log Analytics supported bucket for beyond 30 days. Standard custom log retention pricingwill apply. 

Get started today

Now that Log Analytics is Generally Available, you can upgrade your log buckets to use Log Analytics and know that it’s covered under the Cloud Logging SLA. Upgrade your log bucket today to start taking advantage of Log Analytics. 

If this is the first time you’re hearing about Log Analytics, we’ve got you covered with some materials to get you up to speed. Take a look at our blog the top 10 reasons to get started with Log Analytics, watch a recent on-demand information session we did aimed at developers, and learn more about the overall challenges we’re helping you solve in this video: Streamline software development with better insights and diagnostics.

Editor’s note: just looking to register for the fireside chat on January 31st? Here you go! 

You probably already know that cloud migration continues to be a huge priority for organizations across the globe. But what may surprise you is all the ways priorities, projects, and outcomes have changed in the last few years. For example, in 2020, the thing that was most top of mind was identifying the best-fit workloads to run in the public cloud. But what about now? 

To answer that question (and more) we’re hosting a free ‘fireside chat’ on January 31st at 9:00AM PST where our guest Bill Martorelli, Principal Analyst at Forrester, and Priyanka Vergadia, Staff Developer Advocate at Google Cloud will do a deeper dive on the findings of our “State of Public Cloud Migration” study for 2022. 

During the chat, Bill and Priyanka will discuss key findings from this Google-commissioned Forrester Consulting study, which includes insights from 300+ decision makers and answers questions like: 

Why are more global organizations accelerating their cloud migrations?

What factors do global organizations prioritize when selecting cloud vendors?

What are the IT and business benefits that global organizations gain after migration?

For instance, circling back to our original example, it turns out that identifying best-fit workloads to run in the public cloud is not the top priority anymore. In fact, many organizations have already moved past that phase of their migrations, with their top priorities now focusing on post-migration performance and cost optimization strategies. 

It’s sure to be a great time, and a fantastic way to kick off your new year, especially if cloud migration is one of your New Year’s resolutions. Hope to see you there!