Cloud

API adoption is exploding everywhere — in fact over 80% of web traffic is now API traffic (source). This is because APIs enable easier and more standardized delivery of services and data — and this tends to lead to business growth. Many organizations still regard APIs as implementation detail, but this massive growth of APIs can be difficult to manage without a well-defined API strategy.

This is where API management comes in — making it easier to build APIs, secure every API call, managing your API traffic and productizing critical data assets. At Google Cloud we have three solutions for your API use cases: Apigee API Management, API Gateway, and Cloud Endpoints, and each has a unique sweet spot. So how do you choose which Google Cloud solution is appropriate for your use case? Let’s start with some definitions:

Apigee

Apigee is Google Cloud’s API management product that enables organizations to build, manage, and secure APIs — for any use case, environment, or scale. Apigee lets you operate in any environment of your choice (on-premises, in Google Cloud, or a hybrid environment) with enhanced scale, security and automation. Apigee includes advanced enterprise capabilities including security features like bot and misconfigured API detection, tools to package and monetize digital assets as APIs, flexible runtime support, developer portals, governance policies, and more. Apigee offers two pricing models: Pay-as-you-go pricing to manage your own costs without any upfront commitment, and multiple subscription tiers to maintain predictable costs

When is it optimal?

While Apigee is designed for nearly any use case, it’s especially valuable for customers who have very high API volumes, a need for high reliability and enterprise-grade security, or who rely on APIs exposed to third parties or partners as part of their business. 

Many organizations manage a high and/or growing volume of APIs – often deployed across multi-cloud environments and distributed architectures. This is particularly common in large organizations relying on APIs to modernize their legacy applications without slowing development. In these use cases, Apigee’s flexible deployment options, consistent governance, and robust tools for managing the full lifecycle of APIs are essential to managing the APIs consistently at scale.

Maintaining high API uptime, performance and security in spite of unexpected traffic spikes or growing malicious attacks is also essential for many organizations. Often these are use cases where customer-facing applications rely on APIs to orchestrate large amounts of business-critical or sensitive transactions. Apigee’s comprehensive controls to monitor, secure, and analyze API traffic enable organizations to maintain customer trust and consistent performance.

Finally, many organizations leverage public APIs as a channel to create new revenue streams or collaborate with external partners. Often these use cases focus on maximizing the value of digital assets by packaging them into API products and monetizing their adoption. In these use cases, Apigee’s multiple developer portal offerings, API productization and monetization capabilities streamline this process significantly.

Why do customers love it?

API products – Ability to productize APIs easily (e.g., developer portals) leads to new revenue streams and opportunities

Security & governance – Automated analytics, monitoring and governance help establish consistent standards without slowing down the pace of innovation

Battle tested – Apigee is Google Cloud’s primary API Management platform. Google (Apigee) was recognized as a Leader in the 2021 Gartner® Magic QuadrantTMfor Full Life Cycle API Management six times in a row1 

Flexibility – Apigee can manage home grown or third-party APIs, REST or GraphQL, on premises, in Google Cloud or a hybrid deployment

API Gateway

API Gateway is a fully managed service that enables developers to create, secure, and monitor APIs for services built on Google Cloud. Designed for serverless workloads, API Gateway makes it easy to manage APIs for Cloud Functions, Cloud Run, and App Engine. API Gateway includes security features like authentication and key validation as well as monitoring, logging, and tracing. With consumption-based pricing, it’s also easy to manage costs with API Gateway.

When is it optimal?

API Gateway is optimal for cloud-native use cases where the target backends are generally limited to services deployed on Google Cloud, and where the developer audiences consuming the APIs don’t require a developer portal. Often these are for very specific API use cases to package serverless applications: Cloud Functions, Cloud Run, and App Engine. This is often especially useful for digital-native organizations, small businesses, or for larger organizations that are building new applications or testing a proof of concept. If you’re getting started with API-first driven development and building serverless backends, then using API Gateway is a great way to get started.

Why do customers love it?

Easy to take an OAS v2 spec and simply import it for generation

Seamless API gateways for serverless apps at Google Cloud

Very inexpensive to get started

Cloud Endpoints

Cloud Endpoints is a gateway that enables developers to to configure, deploy and manage an ESPv2 proxy. Although Cloud Endpoints includes basic functionality like user authentication, basic cloud logging and monitoring, and API keys to validate API calls, it requires customers to manage every detail of the API operations. You can manage Cloud Endpoints costs with consumption-based pricing.

When is it optimal?

Cloud Endpoints is most useful when you specifically want to host the gateway on your own runtime with private networking, but would like the same kind of control plane features (analytics and authentication enforcement) that you enjoy with API Gateway.

The most common use case for Cloud Endpoints today is with gRPC services, where a developer wants to locally host the gateway with their project. This can be useful if you’d like to run and test the Cloud Endpoint on your own local machine during development.

So what should you choose?

First of all, if you’re already using one of these products and it meets your needs, keep doing what you’re doing. If not, try using the decision tree below to consider key factors in the decision process.

For most customers running complex or high-scale applications, we suggest considering Apigee first because it’s a full-featured product designed for API consumption across teams, partners, organizations, and even ecosystems. You will never outgrow Apigee because it’s both the most feature-rich and scalable, and also the most cost-effective API management option at high scale. Apigee is where we are investing most heavily in developing new features and functionality — your API product dreams are most likely to come true with Apigee. 

However, if you’re working on a project that leverages serverless products like Cloud Run and you’re not sure if you need to manage third-party APIs, optimize for high scale or establish consistent governance yet, API Gateway is a good choice because it’s a lightweight product designed for exposing APIs at the project level. You can get started slowly and If your organization or project scales up or expands to require new features, you can consider upgrading to Apigee. 

Finally, if you’re running local tests, operating in your own private network, require greater control over your gateway configurations and runtime operations at the cost of operational complexity, then Cloud Endpoints may be a good choice because it’s designed specifically for these use cases. 

Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

1. GARTNER and Magic Quadrant are registered trademarks and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved.

As we help our customers evolve their security approaches for the cloud era, two key objectives based on Google’s experience defending our own infrastructure and systems, emerge: trust nothing and detect everything. These can seem like daunting tasks for even the most capable security teams, especially in today’s challenging threat environment.

But next month, our experts will show you how, with modern approaches and actionable threat intelligence by your side, you can meaningfully strengthen your security posture and increase your resilience to today’s risks and threats.

As you look to secure what matters most in 2023, join us Dec. 7 for our quarterly Security Talks, where you’ll discover guidance on how to advance your prevention, detection, and response capabilities through automation, get an advance look at our latest threat research, and learn how and why Zero Trust should be the foundation for your data protection strategy. You’ll also hear how you can leverage tools you already use today, such as Chrome, to protect your workforce. 

For even more on advances in Chrome security, we encourage you to also check out the Chrome Insider event on Dec. 8.

Another reason to register for Security Talks? This series of digital sessions, created by security practitioners for security practitioners, will kick off with an impactful session by Malcolm “MK” Palmore, director at our Office of the CISO, on the importance of diverse workforces and how they can significantly improve problem solving and overall security performance. Here is the remainder of the agenda, which includes a talk from the Mandiant team, which we recently welcomed to the Google Cloud family: 

Building an advanced detection program for the modern SOC

Controlling access to cloud services in a Zero Trust world

Zero Trust with zero agents: Why Secure Enterprise Browsing should be the foundation of your Zero Trust and data protection strategy

Mandiant Tales from the Front Lines: Activate cyber defenses against supply chain compromise

New Google Cloud Firewall: Big updates you’ll want to hear

Learn how to fight mobile fraud with reCAPTCHA Enterprise

Securing modern apps in Google Cloud

How to collaborate and deploy secure code without trust 

We look forward to sharing our latest security insights and solutions with you. Sign up now to reserve your virtual seat. Security Talks is 100% digital and free to attend. All sessions will be available on demand after they air.

To the casual observer, the Sun is constant and unchanging. But in fact, this nearly perfect ball of plasma regularly spits large quantities of superheated gas into interplanetary space, creating solar storms that wreak havoc on the Earth’s magnetic field as they approach and pass through the atmosphere. These geomagnetic disturbances are visible in polar skies as Aurora Borealis and Aurora Australis, nature’s most spectacular light shows. 

Their beauty notwithstanding, solar storms may have equally dramatic and potentially destructive effects: they can induce extreme voltages in electric wires, threatening the stability of electrical grids, up to and including the possibility of destroying the undersea internet cables that carry much of the world’s internet traffic. 

At Google, we have a long history of building fiber optic cables with our telecommunications partners; to date, we’ve invested in 22 subsea cable projects that deliver services to Google and Google Cloud customers around the world. So needless to say, the prospect of losing any of these cables to a large solar storm got our attention — especially when you consider that Solar Cycle 25, a period of increased solar activity, has begun.

To assess this risk, a team of Google scientists set out to better understand the impact of space weather phenomena on large interconnected systems, and evaluate it against real-world data. What we found is that our subsea cables are safe and that the risk of large-scale cable damage from a solar storm is low. (Phew!) To understand how we came to this conclusion, let’s first shed some light on solar storms and their effect on subsea cables.

Faraday’s law in action

Solar storms provide a large-scale demonstration of Faraday’s law of induction: a variable magnetic field will cause a variable electric field. The Earth’s magnetic field is distorted and weakened as the solar storm approaches, which in turn generates electric fields that propagate through the conducting interior of the Earth. These geoelectric fields can enter grids of transmission lines through their grounding points and induce large voltage swings that can cause problems for their operation. 

This was famously observed in 1859 during a severe solar storm known as The Carrington Event, the most intense geomagnetic storm in recorded history, which caused severe damage to the telegraph network of the time. Contemporary electric grids are at risk too: the March 1989 geomagnetic storm knocked out power across large sections of Eastern Canada. In short, based on Faraday’s law, any interconnected electrical system that is anchored in the Earth is at risk.

Submarine cables fit that description, connecting continents across the sea floor over very long copper lines. The cables consist of glass fibers through which a laser shoots light to the other side. There are also underwater optical amplifiers along the line that boost the light signal at regular intervals.

On both sides of the ocean, a landing station contains the lasers as well as redundant high-voltage powering feed equipment (PFE) to provide power to the many repeaters along the path. The Earth’s ground completes the electrical circuit. For redundancy, the two PFEs maintain a design voltage between themselves. For example, for a cable that needs 10,000 Volts, one end provides +5,000 Volts and the other -5,000 Volts. If one of the PFEs fails, the other is designed to double its voltage so the overall voltage across the cable remains the same. When this happens, the PFE that was previously carrying 5,000 Volts surges to assume the whole load, in keeping with its redundant design. 

A shock to the system

How much voltage would a large solar storm add to a very long copper wire under the ocean? To find out, Google scientists analyzed voltage fluctuations in a number of geographically distributed submarine cables over a five-month observation period that captured both moderate and low solar storm activity. Regardless of the length or orientation of the cable, it turns out that the voltage disturbance was closely correlated to the strength of the geomagnetic perturbation. This allowed us to estimate the impact of a future Carrington-type event without actually waiting for one.

The graph shows the voltage increase on different submarine cables as a function of the storm’s magnetic field fluctuations, recorded by a worldwide network of magnetic observatories. Each data point corresponds to a storm that registered on a submarine cable and its nearest magnetometer. There were no huge storms during our observation period, so none of the storms caused significant voltage variations. 

Luckily for us, a transoceanic cable captured the 1989 storm, inducing a voltage of about 300 Volts above the nominal power of the cable. Extrapolating from these observations, an event like the 1859 Carrington Event would only result in a voltage increase of 800 Volts. Because of their redundant power design, cables such as our transatlantic lines can typically absorb fluctuations of up to 6,000 Volts — much larger than even the 1859 storm. 

So, fortunately, submarine systems appear to be well protected against the effects of solar storms. We will publish the results of our study in an upcoming peer-reviewed scientific paper next year. In the interim, we’re glad to report that, at least as far as submarine internet cables are concerned, you can safely enjoy the northern lights without worrying that they’ll take out the internet.

At Google Cloud Next in October, we announced the Preview release of advanced rule tuning for Cloud Armor which can provide customers more granular control over how they apply our pre-configured Web Application Firewall (WAF) rules. We also announced the Preview of our auto-deploy option for proposed mitigating rules generated by Cloud Armor Adaptive Protection, our machine learning-based attack detection and response capability. 

Advanced rule tuning for pre-configured WAF rules

With the advanced rule tuning capabilities, customers may now: 

Tune preconfigured WAF rules by sensitivity level

Deploy preconfigured WAF rules using signature opt-in or opt-out mode

Exclude request fields from WAF rule inspection 

Advanced rule tuning can help customers make rules more effective and reduce the noise from protected applications which expect traffic that matches the underlying signatures. It is considered a best practice to enable verbose logging to help quickly identify which signatures and fields to omit from a given deployment.

Configuration by Sensitivity Level 

Sometimes normal traffic to protected resources can cause a preconfigured WAF rule to trigger even though the request is safe and well-formed. The result is a false positive. While it doesn’t happen often, when it does it can be annoying and distracting. With the latest release of Cloud Armor, the configuration options and process for rule tuning have been expanded, which can greatly simplify how one can address such cases.  Customers can now specify a sensitivity level, which determines which group of signatures in the preconfigured WAF rule are used to inspect an incoming request, thus alleviating the need to list out each specific signature individually. 

The OWASP ModSecurity Core Rule Set (CRS)groups specific threat signature values into logical groupings called Paranoia Levels (1-4), which Cloud Armor implements as the sensitivity level field in its rule language. Because we added a level 0, there are five sensitivity levels:

Sensitivity level 0: No rules applied – Signature opt-in mode

Sensitivity levels 1-4: Sliding scale of signature sensitivity with level 1 corresponding to the highest fidelity signatures (which are least likely to generate false positives), and level 4 corresponding to the highest sensitivity signatures (which are least likely to generate the chances of a false negative)

The Cloud Armor documentation describes in more detail which signatures are associated with which severity level. 

In general, the higher the sensitivity level, the lower the risk of permitting a malicious request through (false negative). However, high sensitivity increases the chance of noisy alerts without further tuning of the rules. Depending on your defense strategy you may wish to start with a more aggressive posture for especially sensitive applications, such as level 4, and relax the rules based on observed noise from your protected application; however, most customers tend to start with a permissive deployment, such as level 1, and tighten as necessary. 

Sensitivity level 1 example:

Signature Opt-out and Opt-in Mode

To further tune your rules, you can also now opt-in or opt-out of certain signatures in a rule set.  

Opt-out: Allows you to disable a signature that you don’t want to be used in the current rule invocation

Opt-in: The inverse function of opt-out, you select only the signatures that you want Cloud Armor to evaluate

Opt-out example:

This rule applies sensitivity level 3 for the CRS rule for PHP, while excluding the signature owasp-crs-v030301-id933111-php, which is normally part of the level 3 grouping. 

To apply the ‘opt_in_rule_ids’ operation, the sensitivity level has to be explicitly set as 0 (no rules applied).

Opt-in example:

In the opt-in example above, Cloud Armor will only process the two Common Vulnerabilities and Exposures (CVE) signatures specified rather than applying the standard Cloud Armor ‘cve-canary’ rules set which currently contains four signatures. 

The opt-in approach can be helpful in two cases. The first is when most of the signature rules are not going to be processed, so rather than listing a longer set of signatures to exclude, you list only the handful of signatures to evaluate. The second is when you always want to verify newly released signatures before deploying them. Such a case can arise when new signatures are added to an existing rule, such as when a new critical, zero-day vulnerability drives new signatures (as occurred with our response to Log4j). Using opt-in IDs, the new signatures will not automatically be evaluated; you can first test and verify them in your specific environment, and then deploy them when you are ready. 

By combining Common Expression Language (CEL) with preconfigured WAF rules you can customize opt-in/opt-out use cases you need to support if specific URLs need specialized processing, while preserving the inspection of the preconfigured WAF signatures elsewhere.

Combining CEL rule with Opt-out rule:

In the CEL rule example, the rule evaluates the remote code execution CRS rule at a sensitivity level of 3, while excluding one signature value only for requests which attempt to reach the login URL based on the specified regular expression match. 

Optionally exclude request fields from inspection 

There are times when you expect incoming requests to contain values, such as special characters, that could trigger a signature in the preconfigured WAF rules. In these circumstances, you still want Cloud Armor to evaluate the request for possible vulnerabilities. For these situations, you can now configure field level exclusions to preconfigured WAF rules within Cloud Armor policy. 

You can now configure Cloud Armor rules to exclude from evaluation specific fields – request headers, cookies, query parameters and URL paths – based on a string pattern match on the field name. 

You can articulate a string sequence to match on a field name using any one of the following operators:

EQUALS

STARTS_WITH

ENDS_WITH

CONTAINS

EQUALS_ANY

Once you have identified the specific WAF rule and offending signature, you can run a gCloud command to tell Cloud Armor to omit the undesired field from evaluation while processing the rule against the remainder of the request. 

Field exclusion example:

In the example above, the exclusion is configured for Cloud Armor to evaluate incoming requests while omitting headers that begin with “abc” from inspection against two signatures, “owasp-crs-v030301-id941140-xss” and “owasp-crs-v030301-id941270-xss”. All other signatures within the “xss-v33-stable” rule set will still be included in the evaluation if they are included in the rule at priority = 1400. 

Auto-Deploy Adaptive Protection’s suggested rules

Cloud Armor’s Adaptive Protection (video) can automatically detect incoming attacks and can analyze the attack traffic to recommend mitigations in near real time. Cloud Armor mitigated one of the largest known DDoS attacks on record using this functionality. Leveraging Google’s machine learning capabilities, Adaptive Protection can analyze traffic flowing to the protected applications to establish a baseline of good traffic, and can detect potential attacks that deviate from the baseline. Alerts include the dynamically calculated signature of the attack, and Adaptive Protection proposes a WAF rule to mitigate it.

When customers come to trust and rely on Adaptive Protection they generally ask us to help cut the human out of the loop when a mitigating rule is generated. We have been hearing this request this past year and a half. With Adaptive Protection’s new auto-deploy feature (in Preview), customers can now have Cloud Armor automatically deploy the Adaptive Protection generated rules based on the following criteria:

Load threshold: The spike in the load of the attacked application or backend service 

Confidence threshold: The confidence score indicating the likelihood of attack

Impacted baseline threshold: The percentage of baseline traffic that may be affected by the rule

Expiration: How long you want the rule to remain in place

Previously, customers needed to manually review the alert and suggested rule, then deploy the rule. It didn’t take long before customers were convinced of Adaptive Protection’s accuracy, and asked us to automate those deployment steps. With the new auto-deploy feature, Cloud Armor customers can automatically deploy rules with a desired action, like block or rate limit, when the configured criteria threshold values are satisfied. 

Next steps

Cloud Armor’s improved configuration options for WAF rule writing can help increase how effective the rules are, and make them more adaptable to the needs of the applications they’re protecting. These tuning capabilities can help reduce noise and simplify deployment, while ensuring maximal security where appropriate. 

We’ve also helped customers react faster to and reduce the damage from Layer 7 attacks by providing the option for automatic deployment of the Adaptive Protection feature’s recommended blocking rules.  

For more information, consult our documentation, contact your Google Cloud sales representative, or reach out to us in the Google Cloud Platform Community’s security Slack channel.

Since 2007, PrestaShop has helped companies unlock the power of e-commerce through its open-source platform. Over 300,000 merchants worldwide use the PrestaShop platform to grow their business and serve online shoppers.

“Our open-source strategy to ecommerce enablement sets us apart,” says Rémi Paulin, Ph.D., Data Architect at PrestaShop. “Customization is becoming more crucial to retailers, and our open-source platform allows companies to continually evolve their sites and services to stand out from competitors.”

As PrestaShop grew, it wished to derive more value from its data, but the company ran into issues caused by a legacy, siloed architecture that negatively impacted data consistency and accessibility.

Let’s look at how PrestaShop works with Google Cloud and partners Fivetran and Hightouch to gain more control over data, enable a beyond-BI data strategy, and increase employee engagement from less than 10% to more than 40%.

Improving trust in data

Core systems at PrestaShop, including SQL and NoSQL databases, and SaaS Applications, were siloed; each presenting its own data, often captured from different sources such as support tickets, marketing engagement, purchase activity, and product usage. This setup made data overall inconsistent as no single system would contain a source of truth, resulting in many inefficiencies,  poor collaboration across teams, and a reluctance to use data to support key decisions.

“Not long ago, less than 10% of the company regularly relied on data, so we were missing opportunities to make more data-driven decisions,” says Paulin. “Data was underutilized, and people were rapidly losing trust in data.”

PrestaShop set out to design a new architecture to address past challenges, such as lack of data consistency, and improve data accessibility.

“Google Cloud, along with Hightouch and Fivetran, allowed us to build a modern stack to solve these challenges and support our beyond-BI data strategy.”

Building a modern data stack

The first step was to build a robust data ingestion pipeline. After considering several vendors, PrestaShop chose to work with Fivetran to extract data from SaaS applications, including Zendesk, HubSpot, and GitHub, to load into BigQuery. They also use Datastream to stream Change Data Capture (CDC) data from transactional databases into BigQuery in real-time.

“Fivetran and Datastream are no-ops, efficient and highly reliable, and relieve our Data Engineers of management tasks. This brings us a high degree of confidence to build the rest of the stack atop these services,” says Paulin.

PrestaShop relies on several Google Cloud solutions, including Dataflow, and a managed Spark service by Ascend.io, for data transformation. It also uses Looker for its semantic modeling capacities and as a self-serve data platform.

As the company continued on its journey to transform how it manages and benefits from data, it engaged Hightouch to enable data accessibility through activation. Sitting on top of Looker, Hightouch unlocks all data models for operational intelligence. For example, in just a few days, the team built a customer knowledge model combining data from multiple sources and used Hightouch to sync data from the semantic layer to Zendesk via Reverse ETL. This allowed the care team to make more data-informed decisions, speeding up the time to resolve support tickets submitted through Zendesk by 33%. 

“Hightouch feels like a natural extension of Looker and reinforces the position of the semantic data model as the single source of truth,” says Paulin. “It powers a variety of Data Activation use cases, supporting our beyond-BI strategy by providing teams with access to data when and where they need it to improve everyday operations. This has a big impact on the company, bolstering employee trust in available data.”

Becoming data-driven

In less than six months, PrestaShop managed to get the entire data stack up and running, build over 30 data models and engage over 120 employees with a small team of only two Data Engineers.

“Data is now accessible to every stakeholder within the company, regardless of their technical abilities,” says Paulin. 

PrestaShop has already seen much progress in its shift to a more data-driven company and is excited to roll out more self-service intelligence capabilities in the future.

“Google Cloud drives home a culture of simplicity around our data stack, which is essential for us, especially given the small size of our engineering team,” says Paulin. “Fivetran and Hightouch share this culture of simplicity. Together, they offer strong foundations to support our data needs.”

Dashboards, which the company had always had an appetite for, are seamlessly created today. Before moving to Looker, a full-fledged dashboard would take an average of six weeks to develop. Now, it takes less than two days – and a simple dashboard can be created autonomously by business users in as little as 15 minutes.

Furthermore, data usage goes beyond dashboards. Thanks to Looker’s self-service exploration capabilities, many stakeholders can now glean insights surrounding product issues and business opportunities. Thanks to Hightouch, teams can activate their data to make better and smarter operational decisions.

“This is a big leap forward and one of many to come as we continue to add new models, activate our data, and onboard more users,” says Paulin. “Given our global reach and unique approach to e-commerce enablement, we know this is just the start of the great things we can accomplish with Google Cloud, Fivetran, and Hightouch.”

Check out Fivetran on Google Cloud Marketplace, or sign up for a free Hightouch workspace to learn more about what partners can do for your business.