Fuzz testing

Last year at Droidcon Online Xavier Gouchet gave a talk titled: “It’s time to up your test game“. Here he talked about testing and more specifically fuzz testing.

Wikipedia describes fuzz testing as follows:

Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks.

Wikipedia

In his talk he introduced Elmyr:

A Kotlin library providing tools to generate “random” values.

Elmyr github page

Seeing how easy it was to use Elmyr, it got me inspired to start using it in my own projects as well. About a month ago I found some time to get started and I’m loving it!

Adding and using Elmyr turned out to be super easy. It took very little time to get up and running.

What surprised me most is that almost immediately I found a bug in my code due to the random test data that Elmyr produces!

Another thing I really like is that I no longer need to think about what test data to use with my subject under test. All I need to do is ask Elmyr to give me some random data.

If you need Elmyr to support some of your own classes for fuzzing, simply create a new ForgeryFactory, add it to the Forge in your test and you’re done.

Now I wish that I learned about fuzz testing ages ago!

If Elmyr doesn’t do the trick for you, Github hosts tons of other fuzzing libraries.

Retrofit API declaration and inheritance

Retrofit 2.7 introduced Interface inheritance. This is great if you want to separate your API definition from the actual networking implementation. You can now define a “plain” interface as API contract, we’ll call this FooApi, and then have another (Retrofit specific) interface inherit from it and add all the Retrofit sugar.

There seems to be a small catch though to make this fully work. You need to make sure that your concrete Retrofit API does not get obfuscated. If it does, Retrofit will throw the following runtime exception: java.lang.IllegalArgumentException: HTTP method annotation is required (e.g., @GET, @POST, etc.).

Let’s say we created this concrete API declaration and called it FooRetrofitApi. We’ve also wired up all Retrofit annotations so it is able to make requests to your API.

All that is left to do is to exclude your concrete API from obfuscation. Do so by adding the following to your ProGuard rules:

-keep class nl.ansuz.FooRetrofitApi { *; }

Bob’s your uncle!

P.S.

So why do the Retrofit annotations get stripped? I think that is because of the way the code is structured. If you follow clean code principles, you’ll end up referencing the FooApi throughout your project. The concrete FooRetrofitApi is only injected. As a result R8 might think that FooRetrofitApi can be more optimised than is actually the case.

Image debugging

The other day I was asked to help out a colleague who was running into some problems while generating an image.

He would generate the image in an Android app, but when he previewed the generated image, sometimes the image was broken. The bottom part of the image seemed to be missing altogether.

What he would do is the following:

  1. Add some text to a Canvas.
  2. Add some images to the Canvas.
  3. Convert the generated Bitmap to a Base64 encoded PNG String.
  4. Print out the Base64 String.
  5. Copy the Base64 String into another Kotlin app. (This was the easiest way for him, as he was new to Android.)
  6. Write the image to the hard drive of his laptop.
  7. Open the image in an image viewer.

The first thing we checked was what configuration was used to load the image that we added to the composite. Changing the configuration made unfortunately no difference to the outcome.

Next we changed how the image was loaded. Originally a Base64 String was decoded and converted to a Bitmap. We changed this to a bit more Android like way by loading the image from Resources. This made no difference either, the resulting image was still broken.

Right, so it’s not the configuration and not how the image is stored. Maybe it is this particular image that is causing it? Replaced this image with a different image and guess what? Still no luck! The resulting composite image was again broken.

Time to take some drastic measures. Let’s turn off steps of the compositing process until we find what step is causing it. Not adding an image, no luck. Not adding two images, nope! Ok, let’s not add text either, of course not. The only thing left was to not fill the background with a solid colour. That seemed to do it!

Obviously generating an empty image wasn’t an option, so we had to keep looking. Taking another leap, we decided to try and preview the image directly in the Android app. Turning all compositing steps back on and now displaying the image in the app directly. Success!

How was this possible? We had gotten a broken image as a result every single time. Then it (finally) hit me: Logcat has a maximum line length! Anything that exceeds the line length gets truncated. The Base64 encoded String that represented the composite image was naturally quite long, exceeding the Logcat maximum line length and thus getting truncated. As a result we had a broken image no matter what we tried.

The moral of the story: Don’t trust your console to print everything you throw at it.

Google I/O 2021 Developer Keynote

Google I/O 2021 Developer Keynote

Android

  1. Android 12
    1. User safety features: d.android.com/audit
    2. Stand-by bucket
    3. UX improvements
  2. Build beautiful apps, easier
    1. AS + Kotlin + JetPack + JetPack Compose
    2. Android Gradle plugin 7.0
    3. Jetpack
      1. Macrobenchmark (alpha)
      2. DataStore (beta)
      3. etc…
    4. Jetpack Compose
      1. Still Beta, already used in prod apps by loads of big tech comps
      2. 1.0 coming in July
  3. Building across screens
    1. New components to handle screen resizing
    2. Easier Google Assistant (voice) API via app capabilities
    3. More JetPack APIs for small (wearable) screens.
  4. AS Arctic Fox

Web

  1. Power
    1. WeRTC improvements
    2. Maps WebGL integration
    3. More hardware access
  2. Performance
    1. Chrome improvements
    2. Core Web Vitals (report), integrated in various tools (web.dev)
      1. Will be included in search results
  3. Privacy
    1. Privacy Sandbox, no more x-site tracking

Flutter

  1. Flutter 2.2 release
    1. Improved desktop support
    2. null safety by default
    3. Upgraded dev tools
    4. other Google SDK updates for Flutter
    5. Material You support
  2. photobooth.flutter.dev for demo project

Firebase

  1. Accelerate app development
    1. New Firebase Extensions
      1. Stripe
      2. MailChimp
      3. Etc…
  2. High-quality experiences
    1. FB performance monitoring updates:
      1. real-time metrics
      2. redesigned dashboard
      3. Trace table
  3. Engagement
    1. FB Remote Config updates
      1. Personalization feature (alpha)

Machine learning

  1. TensorFlow Hub contains loads of ML models.
  2. g.co/on-device-ml
  3. Mobile
    1. TensorFlow Lite
      1. Model Maker
  4. Web
    1. TensorFlow.js (Lite)
      1. for Microcontrollers
    2. CoLab
  5. Cloud
    1. Vertex AI (managed ML platform)
      1. AutoML
      2. Vertex Pipelines
      3. cloud.google.com/vertex-ai

Google I/O 2021 Keynote

Google I/O 2021 Keynote

Workspace

Smart Canvas part, of Google Workspace.Google Workspace will be available to everyone soon.
LaMDA: (Natural) Language model for dialogue applications. Still in R&D.
TPU v4 announced, organised in “Pods” of 4096 TPUs. Available later this year for Cloud customers.
Quantum computing: create an error-corrected qubit in the coming year(s).Quantum AI campus for qubit computing research.

Safety

Aiming to create a password free future.
Password Manager upgrades:

  1. Easy import tool
  2. Deeper intergration w/ Chrome + Android
  3. Automatic password alerts (for compromised accounts)
  4. Quick fix (for compromised accounts)

All products:

  • Secure by default
  • Private by design
  • You’re in control (of your data)

Helpful information

MUM: Multitask Unified Model. Understands 75+ languages and multiple models (think text, speech, images and video) all simultaneously.
Google Lens, AR, MUM all make information more helpful.
Rolling out “About this result” later this month.
Maps improvements:

  • Live View expansion
  • Detailed street maps
  • A more tailored map
  • Area busyness

Shopping graph: connecting websites, prices, reviews, videos and (brand) SKU & Inventory data.
New photos features:

  • Little patterns
  • Cinematic Photos (improvements)
  • More control

Design + Android

Material You: Personally styled apps. (First on Pixel this fall)
Android 12: “The most personal OS ever”

  • Deeply personal
    • All designs has been overhauled
    • A lot of performance upgrades
  • Private and Secure
    • New privacy dashboard
    • Private Compute Core
  • (Work) Better together
    • Better phone + Chromebook integration
    • TV remote features on phone (for devices running Android TV)
    • Digital car key (launching this fall)
  • Beta 1 is available today!

WearOS

  • Fitbit is now part of Google.
  • The next big thing in mobile computing.
  • Largest update ever:
    • A unified platform (with Samsung)
    • New user experience
    • World-class health and fitness

Health

Use AI to analyse health issues faster.

  • Mammography
  • Tuberculosis diagnosis
  • Skin conditions

Project Starline: next generation (3D) video conferencing

Sustainability

Operate carbon free 24/7 by 2030.
Carbon intelligent computing platform

Droidcon Online 2020

Almost all Droidcon events have been cancelled for 2020 due to COVID-19 and the measures the various countries have taken to battle the outbreak. To be able to continue learning in the Android field, the organizers of Droidcon have set up Droidcon Online.

Through droidcon Online, a new virtual community event series, we intend to keep on doing what we do best, which is creating a safe and fun space for the community to interact and exchange ideas and information.

online.droidcon.com

There are four topics and each will be covered extensively in a series of online talks (roughly) every other week.

  • Jetpack series
  • Multi-platform series
  • Advanced Kotlin series
  • Hands-on & in-depth series

Videos of the talks will be published after they have been presented. Below is a link to each VOD.

Jetpack Series – Part 1 of 3

April 23

Advanced Kotlin Series – Part 1 of 3

April 30

Multi Platform Series – Part 1 of 3

May 14

Advanced Kotlin Series – Part 2 of 3

May 28

Jetpack Series – Part 2 of 3

June 11

Hands-On & In-Depth Series – Part 1 of 4

June 18

Hands-On & In-Depth Series – Part 2 of 4

June 25

Multiplatform Series – Part 2 of 2

July 16

Hand-on/In-depth Series – Part 3 of 6

July 30

Jetpack Series – Part 3 of 3

August 13

Advanced Kotlin Series – Part 3 of 3

August 27

Cucumber & Android

A while ago, the agile/scrum team I was part of was looking for a way to record functional requirements for the app we were working on. After looking around for a bit, we came across Behaviour-Driven Development (BDD) and Cucumber.

Although documentation and automated tests are produced by a BDD team, you can think of them as nice side-effects. The real goal is valuable, working software, and the fastest way to get there is through conversations between the people who are involved in imagining and delivering that software.

Cucumber.io on BDD

Since we wanted to test the Android and iOS using the same Gherkin scenarios, we set up Cucumber together with Appium. Personally I find working with Appium too cumbersome. In a private project I’ve therefore set up Cucumber with Android and left Appium out of the equation.

In this article I’ll explain how to set up Cucumber for Android.

Set up cucumber-android

Add the “cucumber-android” library to your project:

androidTestImplementation "io.cucumber:cucumber-android:$cucumberVersion"

Custom TestRunner

To configure Cucumber we’ll create a custom “TestRunner‘:

private const val PLUGIN_KEY = "plugin"
private const val REPORTS_DIR = "reports/cucumber"

private const val REPORTER_PLUGIN_PATTERN =
    "junit:%s/cucumber-junit.xml--" +
    "html:%s/cucumber-html--" +
    "json:%s/cucumber.json"

/**
 * The CucumberOptions annotation is mandatory for exactly one of the classes in the test project.
 * Creating a custom test Runner seems the simplest way to achieve that.
 */
@CucumberOptions(
    features = ["cucumber/features"],
    glue = ["nl.ansuz.android.test"]
)
class CucumberRunner : CucumberAndroidJUnitRunner() {

    override fun onCreate(bundle: Bundle?) {
        bundle?.putString(PLUGIN_KEY, getPluginConfiguration())
        super.onCreate(bundle)
    }

    private fun getPluginConfiguration(): String {
        val path = getReportsPath()
        return REPORTER_PLUGIN_PATTERN.format(path, path, path)
    }

    private fun getReportsPath(): String =
        File(targetContext.getExternalFilesDir(null), REPORTS_DIR).absolutePath
}

Permissions

To make reporting work, we need to give our app permission to write to the external storage. Since Cucumber will run under the “androidTest” flavor, you can add the extra permissions in “src/androidTest/AndroidManifest.xml” to avoid polluting your production app with unnecessary permissions.

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Now Cucumber can save test reports on the external storage.

Writing Gherkin

The next step is to write some scenarios so there is something for Cucumber to test.
The cucumber.io website comes with plenty of examples, so I’ll leave this as an exercise for the reader.

Note: The “cucumber-android” plugin expects all feature files to be created under “app/src/androidTest/assets“. You can tweak the base path by changing the “features” property of the “CucumberOptions” annotation in the custom “TestRunner” we created earlier.

Running tests

Now that everything is set up and we have a first test, we can run it. Because of the custom test runner, I had to create a Gradle task that will run instrumentation tests with it.

task runCucumberInstrumentationTests(type: Exec) {
     description 'Runs instrumentation tests with Cucumber'
     group 'verification'
     dependsOn 'installDebug', 'installDebugAndroidTest'
     commandLine 'adb', 'shell', 'am', 'instrument', '-w', 'nl.ansuz.android.test/nl.ansuz.android.test.CucumberRunner'
 }

When you run this new task, no tests pass, but the Cucumber reports are generated anyway. To make the tests pass,we have to create some “glue”.

Writing the glue

In the custom “TestRunner” we’ve also defined where the “cucumber-android” library can find the “glue” to match scenarios with step definitions.

The easiest way to write your step definitions is to annotate the appropriate methods in your Espresso tests with Cucumber annotations, e.g.

@When("the Maker starts a game")
fun startGame() {
  ...
}

Pulling Cucumber reports

Now that you have written some tests, can run them and they pass, you want to be able to have a look at the Cucumber report(s) as well. Pulling the reports is fairly simple, all we need to do is add a Gradle task that uses “adb pull” to do this for us.

task pullCucumberReports(type: Exec) {
    description 'Pulls the Cucumber reports from the emulator.'
    group 'verification'
    dependsOn 'runCucumberInstrumentationTests'

    workingDir buildDir
    commandLine 'mkdir', '-p', 'reports'
    commandLine 'adb', 'pull', '/mnt/sdcard/Android/data/nl.ansuz.android/files/reports/cucumber', 'reports'
}

The reports path is set in the “REPORTS_DIR” constant in the custom “TestRunner” from earlier. Make sure that the paths in the test runner and the Gradle task match.

Cleaning up

So far we are able to run Cucumber tests and pull the reports from the device. What is missing is a way to clean up after testing has finished. Again we’ll add a new Gradle task to do this for us.

task uninstallCucumberTest() {
    description 'Uninstalls the debug and debugAndroidTest apps.'
    group 'install'
    dependsOn 'uninstallDebug', 'uninstallDebugAndroidTest'
}

Tying it all together

Because we are good developers and we are lazy, we don’t want to execute three different Gradle tasks to run tests, pull reports and clean up. Let’s introduce one last Gradle task to make our lives even easier.

task cucumberCheck() {
    description 'Installs regular and instrumentation apps, runs Cucumber tests, pulls reports ' +
            'and then uninstall both apps.'
    group 'verification'
    dependsOn 'pullCucumberReports'
}

cucumberCheck.finalizedBy uninstallCucumberTest

I hope this helps you to get you started on using BDD for your Android project with Cucumber.

Bonus: IntelliJ / Android Studio plugins

To make working with Cucumber and Gherkin easier, you can install the following plugins for IntelliJ or Android Studio:

  • “Gherkin” which provides support for the Gherkin language.
  • “Cucumber for Kotlin” which enables Cucumber support with step definitions in Kotlin.
  • “Cucumber for Java” which enables Cucumber support with step definitions in Java.

Further reading

If you want to learn more about Cucumber, I can highly recommend reading “The Cucumber Book” by Matt Wynne and Aslak Hellesøy, with Steve Tooke.

For an introduction into BDD and Gherkin, the cucumber.io website offers a lot of documentation:

Notes

Software versions used at the time of writing:

  • Gradle: 5.2.1
  • cucumber-android version: 4.2.5

Google I/0 2019 keynote

10.00 Sundar Pichai

Keep making magic

Goal: Building a more helpful Google for everyone.
Google News “full coverage” will become a standard result for Google Search.

  • Helpful

10.10 Search

Adding camera & AR to search results. Placing 3D models in your own environment.
Google Lens now part of Assistant, Photos and Camera.

New Lens features:

  • Find menu recommendations in restaurants.
  • Text to speech.
  • Translations in situ.

10.20 Sundar Pichai

Duplex now available in 44 states in the USA.
Adding tasks on the web:

  • Rental car reservations. (Gets context information from GMail.)

10.25 Scott Huffmann – Google Assistant

Google Assistant data on your device. Process requests in near real-time.
Move AI models to the phone, data was reduced from 100 GB to 0.5 GB.
This new assistant will be available on the new Pixel phones later this year.

You are in control of your data in Google Assistant.

New feature: Driving mode. Available this summer.

10.40 Sudar Pichai – Privacy & Security

Building AI for everyone. How to make sure AI models don’t confirm bias? TCAV

Google Privacy controls enhancements

  • Accessible from profile photo. (One tab access to Google Account)
  • Manageable from search.
  • Incognito mode in Maps & Search

Use your Android phone as a security key.

Federated learning, no longer sending data to the cloud, just the model updates. So no user data is being sent.

New feature: Live caption. Runs entirely on device.

New technology: Live Relay.

New research: Project Euphonia. Voice recognition for people with a speech impediment.

11.00 Stephanie – Android

Android Q

  • Innovation
  • Security & Privacy

Innovation

Foldables from multible manufactures. Fold from phone to tablet screen. Screen continuity.

5G supported natively.

Live Captions: Instantly create captions from audio in real-time. Entirely done on device. OS wide feature.

Smart reply: Suggested replies / actions. Works for all messaging apps in Android.

Dark Theme!

Security & Privacy

Google Play Protect.

Privacy settings. All privacy settings in one place.

New location controls (in settings).

Faster security updates. OTA security updates for separate OS modules.

Android OS Framework

Last year: Wind down

Focus mode: disable apps that distract you. Coming in fall on P and Q devices.

Parental controls built into device.

11.10 Rick – Google Home

Putting people first.

Nest – The helpful home

  • Easy for everyone
  • Personal for everyone
  • Works together
  • Respects your privacy

New display: Nest Hub Max. $229

  • Camera + 10″ display.
  • Physical mic + cam off switch.
  • Face match.
  • Google Photos integration.
  • Gesture recognition.

Nest Hub: $129 and also available in NL.

Google Pixel

Pixel 3a and 3a XL, starting at $399.

11.20 Sabrina Ellis – Pixel 3a

The Pixel 3a includes a headphone jack. :p

Maps AR available today on Pixel phones.

11.30 Jeff Dean – Google AI

Building a more helpful Google for everyone.

BERT models to help understand language.

Research & Engineering: Flood forecasting.

Building the ecosystem: Google AI Impact Challenge. Announcing the 20 winners today.

11.35 Lily Peng

Applying deep learning to oncology: Lung cancer.

Google I/O keynote 2018

Pre-keynote game: g.co/worlddraw

10.00 Sundar Pichai

Recap of 2017 and looking ahead.

Opening of Google AI centres around the world. Keeping on developing machine learning and AI:

  • Publishing a paper on medical use of machine learning later in the day.
  • GBoard morse code available later today too.
  • Introducing GMail “smart compose”, rolling out this month.
  • Photos: “Suggested actions.”
  • Using T(ensor) PU 3.0, 8x more powerful than 2.0 from last year.
  • Wavenet: adding 6 new voices.

10.30 Scott Huffman – Google Assistant

Google Assistant on over 500M devices. Supports 30 languages. The aim to get more natural conversations.

  • No need to say “Hey Google” with every query: Continued conversation.
  • Multiple actions.
  • Improving family experience: Pretty please.

10.35 Lilian Rincon

Visual conversation with Google Assistant.

  • Smart displays
  • Food pick-up & delivery
  • Assistant in Navigation in Google Maps.

10.40 Sundar Pichai

AI to help set up a booking system, e.g. making an hair appointment. Google Assistant will call an actual salon and make the appointment for you. Google Duplex.

Working hard to give users back time.

Android Dashboard: Show you where (which app) you spend your time.

New Google news.

10.50 Trystan Upstill – Google News

Keep up with the news you care about

  • Briefing: top stories for you. The more you use it, the better it gets.
    • Switch to “headlines” for news from around the globe.
  • Google Material Theme.
  • Newscasts

Understand the full story

  • Full coverage: Using temporal co-locality. Everyone has access to the same information. Using trusted resources.

Enjoy and support the sources you love.

  • Newsstand
  • Subscribe with Google: Access paid content everywhere. Rolling out in the coming weeks.

Rolling out Google News on Android this week.

11.00 Dave Burke – Android

Ten years since the first Android phone.

Android P

  • AI at the core of the OS
    • Intelligence
    • Simplicity
      • M(achine) L(earning) Kit in FireBase: cross-platform.
      • Navigation update.
      • Simplified volume control
      • Rotation: New rotation button on tab bar.
    • Digital wellbeing – Sameer Samat
      • Dashboard
      • App Timer: Set time limits on apps.
      • Do Not Disturb improvements: Hide notifications.
        • Shush: Turn over phone to enter DND
      • Starred contacts
      • Wind Down

Android P Beta available today: android.com/beta.

11.20 Jen Fitzpatrick – Google Maps

  • ML to extract new addresses from Street View + satellite images.
  • New tab: “For you”, personal recommendations.
  • “Your match” score: ML to help analyse your ratings and comparing to the new recommended place.
  • Create a shortlist.

11.30 Aparna Chennapraganda – AI in camera

Walking navigation AR using VPS: Visual Positioning System.

Google Lens

  • Recognise images.
  • Integrated directly into the camera app.
  • 3 new features:
    • Smart text selections: Recognise words
    • Style match
    • Real-time results

11.40 John Krafcik – Waymo

Fully self-driving cars. Driverless transportation service, starting in Phoenix, AZ this year.

Mission: Building a better driver.

Dmitri Dolgov

Explaining where AI can help self-driving cars. Using TensorFlow and TPUs to train neural networks.

  • Perception: Using deepnet (ML) to classify objects.
  • Prediction: Behaviour prediction, e.g. a car running a red light.
  • Learn new skills: Apply a filter to drive in snow.

11.50 Jen Fitzpatrick

Closing notes:

Keep building good things (for everyone).