Google I/0 2019 keynote

10.00 Sundar Pichai

Keep making magic

Goal: Building a more helpful Google for everyone.
Google News “full coverage” will become a standard result for Google Search.

  • Helpful

10.10 Search

Adding camera & AR to search results. Placing 3D models in your own environment.
Google Lens now part of Assistant, Photos and Camera.

New Lens features:

  • Find menu recommendations in restaurants.
  • Text to speech.
  • Translations in situ.

10.20 Sundar Pichai

Duplex now available in 44 states in the USA.
Adding tasks on the web:

  • Rental car reservations. (Gets context information from GMail.)

10.25 Scott Huffmann – Google Assistant

Google Assistant data on your device. Process requests in near real-time.
Move AI models to the phone, data was reduced from 100 GB to 0.5 GB.
This new assistant will be available on the new Pixel phones later this year.

You are in control of your data in Google Assistant.

New feature: Driving mode. Available this summer.

10.40 Sudar Pichai – Privacy & Security

Building AI for everyone. How to make sure AI models don’t confirm bias? TCAV

Google Privacy controls enhancements

  • Accessible from profile photo. (One tab access to Google Account)
  • Manageable from search.
  • Incognito mode in Maps & Search

Use your Android phone as a security key.

Federated learning, no longer sending data to the cloud, just the model updates. So no user data is being sent.

New feature: Live caption. Runs entirely on device.

New technology: Live Relay.

New research: Project Euphonia. Voice recognition for people with a speech impediment.

11.00 Stephanie – Android

Android Q

  • Innovation
  • Security & Privacy

Innovation

Foldables from multible manufactures. Fold from phone to tablet screen. Screen continuity.

5G supported natively.

Live Captions: Instantly create captions from audio in real-time. Entirely done on device. OS wide feature.

Smart reply: Suggested replies / actions. Works for all messaging apps in Android.

Dark Theme!

Security & Privacy

Google Play Protect.

Privacy settings. All privacy settings in one place.

New location controls (in settings).

Faster security updates. OTA security updates for separate OS modules.

Android OS Framework

Last year: Wind down

Focus mode: disable apps that distract you. Coming in fall on P and Q devices.

Parental controls built into device.

11.10 Rick – Google Home

Putting people first.

Nest – The helpful home

  • Easy for everyone
  • Personal for everyone
  • Works together
  • Respects your privacy

New display: Nest Hub Max. $229

  • Camera + 10″ display.
  • Physical mic + cam off switch.
  • Face match.
  • Google Photos integration.
  • Gesture recognition.

Nest Hub: $129 and also available in NL.

Google Pixel

Pixel 3a and 3a XL, starting at $399.

11.20 Sabrina Ellis – Pixel 3a

The Pixel 3a includes a headphone jack. :p

Maps AR available today on Pixel phones.

11.30 Jeff Dean – Google AI

Building a more helpful Google for everyone.

BERT models to help understand language.

Research & Engineering: Flood forecasting.

Building the ecosystem: Google AI Impact Challenge. Announcing the 20 winners today.

11.35 Lily Peng

Applying deep learning to oncology: Lung cancer.

Google I/O keynote 2018

Pre-keynote game: g.co/worlddraw

10.00 Sundar Pichai

Recap of 2017 and looking ahead.

Opening of Google AI centres around the world. Keeping on developing machine learning and AI:

  • Publishing a paper on medical use of machine learning later in the day.
  • GBoard morse code available later today too.
  • Introducing GMail “smart compose”, rolling out this month.
  • Photos: “Suggested actions.”
  • Using T(ensor) PU 3.0, 8x more powerful than 2.0 from last year.
  • Wavenet: adding 6 new voices.

10.30 Scott Huffman – Google Assistant

Google Assistant on over 500M devices. Supports 30 languages. The aim to get more natural conversations.

  • No need to say “Hey Google” with every query: Continued conversation.
  • Multiple actions.
  • Improving family experience: Pretty please.

10.35 Lilian Rincon

Visual conversation with Google Assistant.

  • Smart displays
  • Food pick-up & delivery
  • Assistant in Navigation in Google Maps.

10.40 Sundar Pichai

AI to help set up a booking system, e.g. making an hair appointment. Google Assistant will call an actual salon and make the appointment for you. Google Duplex.

Working hard to give users back time.

Android Dashboard: Show you where (which app) you spend your time.

New Google news.

10.50 Trystan Upstill – Google News

Keep up with the news you care about

  • Briefing: top stories for you. The more you use it, the better it gets.
    • Switch to “headlines” for news from around the globe.
  • Google Material Theme.
  • Newscasts

Understand the full story

  • Full coverage: Using temporal co-locality. Everyone has access to the same information. Using trusted resources.

Enjoy and support the sources you love.

  • Newsstand
  • Subscribe with Google: Access paid content everywhere. Rolling out in the coming weeks.

Rolling out Google News on Android this week.

11.00 Dave Burke – Android

Ten years since the first Android phone.

Android P

  • AI at the core of the OS
    • Intelligence
    • Simplicity
      • M(achine) L(earning) Kit in FireBase: cross-platform.
      • Navigation update.
      • Simplified volume control
      • Rotation: New rotation button on tab bar.
    • Digital wellbeing – Sameer Samat
      • Dashboard
      • App Timer: Set time limits on apps.
      • Do Not Disturb improvements: Hide notifications.
        • Shush: Turn over phone to enter DND
      • Starred contacts
      • Wind Down

Android P Beta available today: android.com/beta.

11.20 Jen Fitzpatrick – Google Maps

  • ML to extract new addresses from Street View + satellite images.
  • New tab: “For you”, personal recommendations.
  • “Your match” score: ML to help analyse your ratings and comparing to the new recommended place.
  • Create a shortlist.

11.30 Aparna Chennapraganda – AI in camera

Walking navigation AR using VPS: Visual Positioning System.

Google Lens

  • Recognise images.
  • Integrated directly into the camera app.
  • 3 new features:
    • Smart text selections: Recognise words
    • Style match
    • Real-time results

11.40 John Krafcik – Waymo

Fully self-driving cars. Driverless transportation service, starting in Phoenix, AZ this year.

Mission: Building a better driver.

Dmitri Dolgov

Explaining where AI can help self-driving cars. Using TensorFlow and TPUs to train neural networks.

  • Perception:¬†Using deepnet (ML) to classify objects.
  • Prediction:¬†Behaviour prediction, e.g. a car running a red light.
  • Learn new skills: Apply a filter to drive in snow.

11.50 Jen Fitzpatrick

Closing notes:

Keep building good things (for everyone).

Using SSL right on Android

General information
There is no out of the box solution from Google to use stronger encryption for the communication between your Android client and a server.

Google provides some information on how to use SSL on the Android developer website titled “Security with HTTPS and SSL“. This site contains two warnings you should not forget.

  1. Do not use a TrustManager that does nothing.
  2. Replacing the HostnameVerifier can be very dangerous.

A reason to use a TrustManager that does nothing could be for local or Alpha testing where you don’t have a valid SSL certificate on the server. Another could be to give QA the option to debug network traffic. If for some reason you do use a TrustManager that does nothing, make sure this implementation can never make it to a production version of the application. One way to do that is only include the empty TrustManager in one of the (testing) application flavors.

If you do want to use a custom HostnameVerifier, use composition to reuse the system’s default HostnameVerifier. This way you can add your own verification and still benefit from the default verification. You should apply the same when building your own TrustManager, that way your validation is never worse than the default system validation.

Pinning certificates
One way to make the encryption stronger is by using pinned certificates. This will help your application protect itself from fraudulently issued certificates. I will explore two ways of pinning a certificate. One is including the actual certificate, the other is to use SPKI or Subject Public Key Info.

Getting the server certificate
Before getting started with either method for pinning a certificate, you need to get hold of the server certificate. The easiest way to optain that would be to ask your system administrator. If that is not possible, you could use OpenSSL or FireFox to just save it from the browser. For more details, see “How to save a remote server SSL certificate locally as a file” on SuperUser.com.

Including the certificate
Start by including the certificate file (as raw resource) in your APK.
After this, implement your own TrustManager that uses the following pseudo-code to analyse the received certificate.

Get the key from the bundled certificate.

  1. FileInputStream fis = new FileInputStream("path_to_pem");
  2. CertificateFactory cf = CertificateFactory.getInstance("X.509");
  3. java.security.cert.Certificate c = cf.generateCertificate(fis);
  4. System.out.println(c.toString());
  5.  
  6. // will get you the public key object you can compare against.
  7. RSAPublicKey pk = (RSAPublicKey) c.getPublicKey();
  8.  
  9. System.out.println(pk.toString());
  10. String s = new BigInteger(1, pk.getEncoded()).toString(16);
  11. System.out.println(s);

Get the server key.

  1. // that gets the public key provided by the server out of the chain.
  2. RSAPublicKey pubkey = (RSAPublicKey) chain[0].getPublicKey();

Compare the two keys.

  1. // Where pk is the one loaded from your app file space.
  2. pubkey.equals(pk);

SPKI
An alternative approach to bundling the certificate with your application is to use the Subject Public Key Info method. This will just store the certificate’s fingerprint instead of a whole file.

To generate the fingerprint for your certificate, follow the instructions in “Appendix A” of the “Public Key Pinning Extension for HTTP” document, example:

  1. openssl x509 -noout -in certificate.pem -pubkey | \
  2.     openssl asn1parse -noout -inform pem -out public.key
  3. openssl dgst -sha256 -binary public.key | openssl enc -base64

You can calculate the same fingerprint in Java using the snippet below:

  1. // Assume the 'certificate' is a X509Certificate.
  2. MessageDigest digest = MessageDigest.getInstance("SHA-256");
  3. byte[] encodedKey = certificate.getPublicKey().getEncoded();
  4. byte[] hash = digest.digest(encodedKey);
  5. String fingerprint = Base64.encodeToString(hash, Base64.NO_WRAP);

Notes
SSL certificates can be invalidated for various reasons (they expire, get compromised, etc). Because of this you should consider showing a message to the user when pinning a certificate fails.

To avoid your app breaking when a certificate gets invalidated, you can keep a list of valid public keys. Start with the first in the list, if that doesn’t work fall back to the next public key in the list.

Update or change a certificate before the previous one expires. Unless the certificate is compromised, the public key will most likely stay the same. (If you use the same private key/CSR.)

Loading Contact photos with Picasso

The Picasso image downloading library has an interesting feature that I just stumbled upon. It can load Contact photos based on a passed in Contact Uri containing the ContactsContract.Contacts.LOOKUP_KEY. It is as simple as:

  1. Uri uri = Uri.withAppendedPath(
  2.     ContactsContract.Contacts.CONTENT_LOOKUP_URI, lookupKey);
  3. Picasso.load(uri).into(myImageView);

It doesn’t seem to be mentioned on the Picasso website, but it is a very useful feature.

If you want to know exactly how Picasso loads the Contact photo, have a look at the ContactsPhotoBitmapHunter class.