Custom Flutter build step: syncing Firebase data on every release

Recently needed to solve a task that required implementing a custom Flutter build step. Client I’m working with wanted the app to be fully functional offline, even if you turn on airplane mode before first launch. This is pretty interesting because it’s taking the offline-first approach a bit further. Usually offline-first apps still require an initial connection to download the necessary data.

Imagine you are storing and updating some public data in Firebase Firestore and your app reads that data from Firebase, using it as backend. Firestore SDK supports offline persistence so most of the time offline access will work out of the box. However, it won’t work on first launch since Firestore needs to download and cache the data first.

This means we should prepackage a snapshot of the db (excluding sensitive parts) with the app. Ok, sounds simple enough: copy a json file into the assets folder, then use it as a fallback if Firestore returns an error.

But we don’t want that snapshot to get too stale and ideally we should update it every time we build a new release. But we also don’t want to do it manually every time and this is where things get more interesting: we need to write a custom Flutter build step that would automatically copy relevant paths from Firestore and save it to assets.

For a custom build step we need something we can run from terminal. Node was my first choice because there is a Firebase Admin SDK for Node and I had already used it when writing custom cloud functions, so I had some code I could reference.

First you need to go to the Firebase console and create a service account in project settings, this will give you a json file to download.

The script itself is fairly simple, we initialize the sdk, read the relevant docs from Firestore, build a json and save it into the assets folder, which will later be packaged with the app:

import * as admin from "firebase-admin";
import * as fs from "fs";
import * as path from "path";

// path to your service account json
const serviceAccount = require("../firebase-service-account.json");

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
});

const db = admin.firestore();

async function updateData() {
  // read relevant collections and docs
  const docRef = db.collection("collection").doc("doc");
  const doc = await docRef.get();

  if (!doc.exists) {
    throw new Error("Document doesn't exist");
  }

  const data = doc.data();

  if (!data) {
    throw new Error("Document is empty");
  }

  // build json
  const jsonData = {
    foo: data.foo,
  };

  // write the json file to assets/data folder
  const assetsDir = path.join(__dirname, "../../assets/data");
  const filePath = path.join(assetsDir, "data.json");

  if (!fs.existsSync(assetsDir)) {
    fs.mkdirSync(assetsDir, { recursive: true });
  }

  fs.writeFileSync(filePath, JSON.stringify(jsonData, null, 2), "utf8");

  console.log(`Data saved to ${filePath}`);
}

updateData();

Let’s put this into the data_sync_tool folder in project root and init Typescript there. cd into the folder, compile with npx tsc and run node dist/index.js from terminal to make sure it works. The result should be a data.json file in assets/data. The exact commands may differ, depending on your setup.

Now to the interesting part – running this script automatically every time we build a release ipa (or aab). It should not run in debug builds.

The process for iOS turned out to be fairly straightforward: open the Runner in Xcode, navigate to Build phases, click the + icon and select “New Run Script Phase”.

Paste:

if [ "$CONFIGURATION" == "Release" ]; then
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
    nvm use default
    node ../data_sync_tool/dist/index.js
fi

Note nvm should be installed for this to work. Now if you build a release ipa, it will have the latest snapshot of your Firestore db. Comment out the if on the first line for easier testing in debug builds.

See documentation for more details https://developer.apple.com/documentation/xcode/running-custom-scripts-during-a-build

Android turned out to be a bit trickier. Open the build.gradle file in android/app folder and paste the gradle task that will run the script:

tasks.register('syncDataTask', Exec) {
    commandLine 'node', '../../data_sync_tool/dist/index.js'
}

Now we need to make sure this task runs before we build our Flutter app. First thing I tried was adding

preBuild.dependsOn syncDataTask

When I built the app (debug or release), it did update the file but I noticed the updated file didn’t actually make into the app. It seemed like it ran too late in the build process and Flutter did not pick up the change.

Luckily, when you build the android version it prints out the list of gradle tasks that are being executed, so it was a matter of finding the right task to depend on.

After some trial and error, I found that depending on compileFlutterBuildRelease worked and the updated file successfully made it into the app:

tasks.configureEach { task ->
    if (task.name == "compileFlutterBuildRelease") {
        task.dependsOn syncDataTask
    }
}

Now when you build a release Android aab/apk, the script will execute successfully.

More details on gradle tasks: https://docs.gradle.org/current/userguide/tutorial_using_tasks.html

Now that we have a working custom build step for both iOS and Android locally, we can also make sure it works in CI/CD. For this we just need to make sure Node is installed. Add this to your .yml file:

- name: Set up Node.js
  uses: actions/setup-node@v2
  with:
    node-version: '18'

- name: Install Node.js dependencies
  run: npm install
  working-directory: data_sync_tool

- name: Compile TypeScript
  run: npx tsc
  working-directory: data_sync_tool

Now everything should be working. By following these steps, you’ve integrated a custom build step that prepackages latest Firestore data with the app. You can use it as a fallback when there is no internet connection and no newer data is cached.