Category Archives for Cloud computing

Google Cloud Messaging Update Boosted by XMPP

Google’s huge update on its Cloud Messaging service is the biggest announcement from this year Google I/O, as it mixes our two core strengths. We strongly believe this is a big deal for mobile developers and we will explain why.

What is Google Cloud Messaging (GCM) ?

google-io-keynote-cloud-1

In case you never heard about it, here is a quick overview.

Google Cloud Messaging have been announced last year as a replacement of Google Cloud to Device Messaging. This is a service that allows mobile developer to notify mobile devices (but also Chrome browser) about important changes on the server-side component of the application. It makes it possible for the device to stay up to date, without the need to use polling. Polling is about checking periodically for updates on your server and is bad for several reasons:

  • it consumes lot of battery, periodically waking up the mobile device network stack.
  • it consumes a lot of useless resources on the server as most of this resource lead to a “no update” reply.
  • it introduces latency as the request for new data can happen long after the data availability (unless you put a very low polling interval, which is even worse for the mobile battery). You can get your news alert hours after every one already knows about it, which defeat the purpose of the alert in the first place.

Push notifications avoid all those drawbacks, saving battery life, providing lower latency, reducing server load. As a permanently connected session to Google servers, it get the important notification messages as soon as they happen, in a battery efficient way.

Being a service that runs at device level, it can be optimized across applications running on the same device and perform further battery saving benefits.

So, this is a really critical feature to implement in application relying on data coming from the network (which actually covers a lot of applications).

GCM offers nice outstanding features like:

  • Support two use cases:
    • “send to sync”: short notification to tell client to get update from developer server.
    • “send data”: larger notification containing the data payload, avoiding an extra client to developer server roundtrip.
  • Time to live can be used to destroy message without notification if device was not online before its expiration. Pending messages can also be replaced with the collapse feature.
  • Delay when idle can be use for less urgent notifications. Device will get it when becoming active again.
  • Multicast messages to send the same notification to up to 1000 devices.

What does GCM update brings to developers ?

The new update is a brand new service deployment bringing a large set of new features for developers.

In short, it brings:

  • Persistent connections are supported between developer backend and Google, allowing sending a larger number of notifications faster.
  • Upstream messaging: This allows the device to send back notifications to the developer server, through Google platform.
  • Notifications synchronization between devices. Basically, this allows a developer to remove a notification from a device when it has been read / processed on another device.

GCM Cloud Connection Service (CCS): Persistent connections using XMPP

Google choose to use XMPP to allow developers to keep a persistent connection. It works as follow:

  • You open an XMPP client connection to Google XMPP GCM server (Port 5235 on gcm.googleapis.com).
  • You can then keep the secure connection open for as long as you wish.
  • You can then send your usual GCM JSON payload in XMPP message packets.
  • This is streaming so you receive possible errors asynchronously and you can match it to original push thanks to XMPP message id.

This alone brings you a huge performance increase, allowing your server to send up to 4000 messages per second on the persistent connection. Knowing you are allowed up to 10 connections, you can possibly send many notifications fast (up to 40k notifications per second).

A few things to note:

  • You cannot use multicast messages with XMPP persistant connection (at least yet).
  • However, you can mix HTTP connexions and XMPP persistant connections in your use case to optimize the performance depending on the use case.

Upstream messaging

Upstream messaging allows your client to send data asynchronously to your server. Compared to HTTP posts, it offer several advantages:

  • It is easy for client developer, with a simple gcm.send command. This is “fire and forget”. Client developer sends the data. Android GCM framework save it locally, with the commitment to deliver it.
  • It is reliable with reliability handled transparently by GCM mobile framework on the client. There is acknowledgement mechanism on the client and between GCM and your server that ensure no message can be lost.
  • Timing to send is optimized across mobile applications. This allow significant battery saving for messages that are not very time sensitive. Optimization is even performed based on the server side down stream needs. If client or server send messages first, the other direction queue is flushed at the same time.
  • It supports time to live. If the message could not be send, because network is unavailable and is not relevant anymore, it is discarded.

As a developer, your server will receive the message through an XMPP connection. However, be very careful about your server efficiency: you have to be robust and read data fast as the GCM server will queue for you 100 messages before starting to overwrite them (and if you are offline for 24 hours it will discard your messages).

Multi-devices notification synchronisation

The goal of this feature is to makes your notifications state becomes up to date and consistent across a user devices, by propagating state change between the user application installations on your various devices.

This feature is designed to bring many added benefits in the way multiple devices are handled.

First, to know that your app is used by the same users, you can group your device Registration Ids under the same notification_key. Notification key is typically a hash of the username. You are allowed up to 10 devices linked to that notification_key.

Once you have done that, you can use that notification_key as a user id to send notifications to all devices of a given user. By doing so, you let GCM performs lots of optimizations under the hood. Notification will be send in priority to the active device or the last active device. Other devices will get the notification a bit later in a delay while idle type of delivery.

Once the notification has been processed and dismissed, other devices are notified and they can remove the notification as well. If they were not notified yet, notification is directly canceled from the notification queue.

Conclusion

As you see, update to Google Cloud Messaging is really a big update for developers, increasing the number of situation this platform is relevant.

We are already working on supporting those improvements on our push notification platform to help you all benefits of those improvements.

Stay tuned for more information on this support very soon.

(Source: Process-one.net)

Google I/O 2013: Services, services, services

Today was the keynote of Google I/O developer conference. The keynote is usually the place where major announcements are made regarding the Google ecosystem.

Despite impressive announcements, the most important thing that strikes me is not what has been released, but what has not been mentioned.

Services, services and more services

First, what are the main areas of focus this year ?

Either on Android, on Chrome or on the Cloud and server engine architecture Google is showing its consistency in pushing further existing services, adding new ones, and integrating all the pieces together. Here is the impressive list of highlights to their service stack:

  • Google Maps API V2 and location API improvements, with:
    • Fused location = faster, more accurate and more battery friendly.
    • Geofencing, ability to save up to one hundred location triggers per application.
    • Activity recognition based on the phone accelerometer. Device can know if your are walking, cycling, walking, driving. This is a battery efficient, not relying on GPS.
  • Google+ Sign in, brings deep integration between website and Android apps with Google+ service.
  • Google Cloud Messaging, Google Push Notification service, with three major highlights:
    • Persistent connections are supported between developer backend and Google, allowing sending a larger number of notifications faster.
    • Upstream messaging: This allows the device to send back notifications to the developer server, through Google platform.
    • Notifications synchronization between devices. Basically, this allows a developer to remove a notification from a device when it has been read / processed on another device.

Google Cloud Messaging is one our main area of interest. We are already working on the new features and we will have announcements to make soon under our mobile Boxcar brand. Stay tuned :)

  • Google Play Game Service, with:
    • Cloud Save to synchronize your game progress across devices.
    • Achievements and Learderboard, integrated with Google+.
    • Multiplayer API to help developer with networking part.
    • Matchmaking to find players to play with.
    • Cross-platform experience on Android and iOS.
  • Google Wallet: low profile (aka not really promoted in the keynote) improvements, like GMail payments, or easier checkout on mobile.
  • Better developer console to analyse how Android apps are doing and optimize their performance, more specifically:
    • Optimization tips
    • App Translation Service.
    • Referal tracking
    • Usage metrics (Google Analytics from the developer console).
    • Revenue graph
    • Beta testing and stage rollout management

There have also been announcements focusing on Google services improvements or addition for main users (as opposed to developers):

  • Google Play Store improvements to better promote apps to users.
  • Google Music subscription service (US only for now).
  • Huge Google Maps rework and redesign.
  • Search improvements with focus on:
    • more knowledge graph integration in search.
    • more integration of personal information and Google+, circle based personnalisation in search results.
    • Better conversation (aka iteratively refined voice search), with conversation-based queries coming to Chrome on the desktop.
    • Google Now improvements with more cards, to anticipate your search needs on the go.
  • Google+ improvements, mostly for end users:
    • Stream redesign
    • Hangouts chat system, which is a cross platform merge of all Google chats. It is cross-platform, focus on conversation, realtime, photo sharing and video group calling.
    • Photos management, with impressive auto-enhancement and sorting features.

Note from an XMPP developer perspective: Does the new Hangouts mean Gtalk and XMPP will disappear, along with interoperability ? There was not word about it, but I think so.

Impressive, isn’t it ?

Still, as a developer, that first day strangely leaves me with a feeling of unfullfilled expectations. Why ? I think to understand it, we need to list what Google did not talk about.

What Google did not talk about

In previous Google I/O, the center stage is usually taken by:

  • Android updates: There was none announced today.
  • Shiny new devices, usually prerelease to developers. Nothing on this part as well.
  • New unexpected projects, like Chrome, Google Glasses, Google TV, or even the now dead Wave.

On this side, nothing has been announced. No mention of Android for home or TV. No successor for the now dead Nexus Q. No update on Android Accessory Developement Kit. No glasses push. No new wearable computer.

Despite a few talks on Google glasses tomorrow, there have been little mention of the progress so far in the keynote.

This year, Google is focusing on services for several (valid) reasons:

  • Those services are updated directly on the devices through Google Play Store. They can more easily push the updates to the end users.
  • Services are perceived are Apple’s Achille heel.
  • Services are a way to put Google at the front stage and differenciatethe Google experience from the various Android forks. It also allow Google to differenciate from device manufacturers that are increasingly trying to get the front stage with their Android devices.

But as Google focus on Services, the story they tell is increasingly about themselves.

For Android developer, on the most major highlight (outside of Google Services) was Android Studio, a development environment based on Jetbrains Intellij, release today in early preview (version 0.1!).

Google have even been heavily promoting Chrome on Desktop, but now also on Android and iOS, focusing on bringing the same experience from all the environment. Along with the fact that both Chrome and Android and under the unique direction of Sundar Pichai, this leave a strange confusing impression.

My conclusion is that for Google, devices do not matter. When Larry Page says that he wants the technology, the device to disappear, he actually means it in the proper sense. Google Glasses and conversational search are a steps in that direction. They are the most straightforward access to Google services. Ideally, they should not even be needed.

It does not matter if you use Chrome, Glasses, Android or iOS to access Google Services. What matter are the services themselves and the contextual data that can be gathered to improve relevance and personnalisation of the service.

Sundar Pichai said two days ago that Google I/O will not be centered on the devices. It is because devices are not an end but a mean for Google.

I feel at this Google I/O, the goal of Google has never been more clear (if you look through the confusion I mentioned earlier).

At this very moment, the path of Apple and Google may split there:

Google wants to improve people lifes with services, making the technology totally hidden. Apple wants to improve people life by focusing on how people interact with the technology (touch, voice, and more). This goes through devices improvements (lighter, faster, easier to use), not making the devices disappear.

Today, I feel that we are at a turning point, I am really looking forward WWDC to see what will be Apple move.

(Source: Nati Shalom’s Blog)

Amazon S3 – Two Trillion Objects, 1.1 Million Requests / Second

Last June I blogged about the first trillion objects stored in Amazon S3. On the first day of re:Invent I updated that number to 1.3 trillion.

It is time for another update!

I’m pleased to announce that there are now more than 2 trillion (2 x 1012) objects stored in Amazon S3 and that the service is regularly peaking at over 1.1 million requests per second.

It took us six years to grow to one trillion stored objects, and less than a year to double that number.

What Does That Mean?
It is always fun to try and put these numbers into real-world terms:

I spoke at a cloud computing conference in China last week. With a population of 1.35 billion, there are 1,481 Amazon S3 objects per Chinese citizen!

Our galaxy is estimated to contain about 400 billion stars. That works out to five objects for every star in the galaxy.

The field of Paleodemography estimates that 100 billion people have been born on planet Earth. Each of them can have 20 S3 objects.

And one more — our universe is about 13.6 billion years old. If you added one S3 object every 60 hours starting at the Big Bang, you’d have accumulated almost two trillion of them by now.

(via Amazon Web Services)