Move large test assets from APK to adb push#17721
Conversation
vit_b_16.pte (~330MB) and stories.pte (~29MB) are no longer bundled in the androidTest APK. Instead they are uploaded as a separate CI artifact and pushed to /data/local/tmp/executorch/ via adb before tests run. TestFileUtils.prepareTestFile() checks the push directory first and falls back to APK resources, so both CI and local workflows remain functional. This reduces the APK from ~361MB to ~2MB.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17721
Note: Links to docs will display an error until the docs builds have been completed. ❌ 8 New Failures, 3 Unrelated FailuresAs of commit fc8d24b with merge base ac3ea74 ( NEW FAILURES - The following jobs have failed:
FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This PR needs a
|
…st-assets-to-adb-push
Golden artifacts (mobilenet_v2, vit_b_16 models and their input/output bins) are no longer downloaded at build time or bundled in the test APK. Instead, they are downloaded and pushed via adb at test runtime, keeping the build step faster and the APK smaller.
stories.pte and tokenizer.bin are now downloaded directly in run_android_emulator.sh and pushed via adb, same as golden artifacts. This eliminates the push-artifacts upload/download steps from CI and simplifies android_test_setup.sh to only export XOR models.
…st-assets-to-adb-push
There was a problem hiding this comment.
Pull request overview
This PR optimizes Android test APK size by moving large test assets (~359MB total) from APK bundling to adb push during CI test runs. The changes introduce a dual-source test file resolution strategy that checks adb-pushed files first before falling back to APK resources, reducing the test APK from ~361MB to ~2MB while maintaining compatibility with both CI and local development workflows.
Changes:
- Added adb push workflow in CI to download and push large test models (stories.pte, vit_b_16.pte, and golden artifacts) from S3
- Created
prepareTestFile()utility that checks/data/local/tmp/executorch/first, then falls back to APK resources - Migrated ModuleE2ETest and LlmModuleInstrumentationTest to use the new dual-source file resolution
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| scripts/run_android_emulator.sh | Added curl downloads and adb push commands for large test artifacts (stories, tokenizer, golden models) |
| extension/android/executorch_android/src/androidTest/java/org/pytorch/executorch/TestFileUtils.kt | Added prepareTestFile() method that checks adb push directory before falling back to APK resources |
| extension/android/executorch_android/src/androidTest/java/org/pytorch/executorch/ModuleE2ETest.kt | Migrated from direct resource loading to prepareTestFile(), removed unused imports, simplified file loading logic |
| extension/android/executorch_android/src/androidTest/java/org/pytorch/executorch/LlmModuleInstrumentationTest.kt | Migrated from manual resource extraction to prepareTestFile(), removed unused imports and file handling code |
| extension/android/executorch_android/android_test_setup.sh | Removed prepare_tinyllama() and prepare_golden() functions as large models are now handled via adb push in CI |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Instead of extracting the entire golden zip and pushing all files, explicitly list the 6 files required by ModuleE2ETest and only extract/push those.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| BASEDIR=$(dirname "$(realpath $0)") | ||
| RESOURCES_DIR="${BASEDIR}/src/androidTest/resources" | ||
| mkdir -p "${RESOURCES_DIR}" | ||
|
|
||
| prepare_xor() { | ||
| pushd "${BASEDIR}/../../training/" | ||
| python3 -m examples.XOR.export_model --outdir "${BASEDIR}/src/androidTest/resources/" | ||
| mv "${BASEDIR}/src/androidTest/resources/xor.pte" "${BASEDIR}/src/androidTest/resources/xor_full.pte" | ||
| python3 -m examples.XOR.export_model --outdir "${BASEDIR}/src/androidTest/resources/" --external | ||
| python3 -m examples.XOR.export_model --outdir "${RESOURCES_DIR}/" | ||
| mv "${RESOURCES_DIR}/xor.pte" "${RESOURCES_DIR}/xor_full.pte" | ||
| python3 -m examples.XOR.export_model --outdir "${RESOURCES_DIR}/" --external | ||
| popd | ||
| } | ||
|
|
||
| prepare_tinyllama() { | ||
| local S3_BASE="https://ossci-android.s3.amazonaws.com/executorch/stories/snapshot-20260114" | ||
| curl -C - -Ls "${S3_BASE}/stories110M.pte" --output "${BASEDIR}/src/androidTest/resources/stories.pte" | ||
| curl -C - -Ls "${S3_BASE}/tokenizer.model" --output "${BASEDIR}/src/androidTest/resources/tokenizer.bin" | ||
| } | ||
|
|
||
| prepare_golden() { | ||
| local url="https://gha-artifacts.s3.amazonaws.com/pytorch/executorch/test-backend-artifacts/golden-artifacts-xnnpack/golden_artifacts_26022500.zip" | ||
| curl -sL -o /tmp/golden.zip "$url" | ||
| unzip -o /tmp/golden.zip -d /tmp/golden/ | ||
| for model in mobilenet_v2 vit_b_16; do | ||
| cp "/tmp/golden/xnnpack/${model}.pte" "${BASEDIR}/src/androidTest/resources/" | ||
| cp /tmp/golden/xnnpack/${model}_input*.bin "${BASEDIR}/src/androidTest/resources/" | ||
| cp /tmp/golden/xnnpack/${model}_expected_output*.bin "${BASEDIR}/src/androidTest/resources/" 2>/dev/null || echo "Warning: no expected_output files for ${model}" | ||
| done | ||
| } | ||
|
|
||
| prepare_xor |
There was a problem hiding this comment.
android_test_setup.sh no longer downloads/prepares the TinyLlama (stories.pte/tokenizer.bin) and golden XNNPACK artifacts into src/androidTest/resources. With the current tree, that resources directory contains only test.txt, so local connectedAndroidTest runs (without the CI adb push step) will fail even though prepareTestFile() claims it can fall back to APK resources. Either keep an opt-in download step for local workflows, or update the local test instructions/README to reflect that these tests now require pre-pushing assets.
| S3_BASE="https://ossci-android.s3.amazonaws.com/executorch/stories/snapshot-20260114" | ||
| curl -sL -o /tmp/stories.pte "${S3_BASE}/stories110M.pte" | ||
| curl -sL -o /tmp/tokenizer.bin "${S3_BASE}/tokenizer.model" | ||
| adb push /tmp/stories.pte /data/local/tmp/executorch/ | ||
| adb push /tmp/tokenizer.bin /data/local/tmp/executorch/ |
There was a problem hiding this comment.
The artifact downloads use curl -sL without --fail/--fail-with-body, so an HTTP error (e.g., 403/404) can still exit 0 and produce an HTML error page that then gets adb push’d. This makes CI failures hard to diagnose and can lead to corrupted test inputs. Consider adding curl -f (or --fail-with-body), plus a small retry/backoff, and optionally validating file size/checksum before pushing.
| adb shell mkdir -p /data/local/tmp/executorch | ||
|
|
||
| S3_BASE="https://ossci-android.s3.amazonaws.com/executorch/stories/snapshot-20260114" | ||
| curl -sL -o /tmp/stories.pte "${S3_BASE}/stories110M.pte" | ||
| curl -sL -o /tmp/tokenizer.bin "${S3_BASE}/tokenizer.model" | ||
| adb push /tmp/stories.pte /data/local/tmp/executorch/ | ||
| adb push /tmp/tokenizer.bin /data/local/tmp/executorch/ | ||
|
|
||
| GOLDEN_URL="https://gha-artifacts.s3.amazonaws.com/pytorch/executorch/test-backend-artifacts/golden-artifacts-xnnpack/golden_artifacts_26022500.zip" | ||
| GOLDEN_FILES=( | ||
| mobilenet_v2.pte | ||
| mobilenet_v2_input.bin | ||
| mobilenet_v2_expected_output.bin | ||
| vit_b_16.pte | ||
| vit_b_16_input.bin | ||
| vit_b_16_expected_output.bin | ||
| ) | ||
| curl -sL -o /tmp/golden.zip "$GOLDEN_URL" | ||
| unzip -o /tmp/golden.zip "${GOLDEN_FILES[@]/#/xnnpack/}" -d /tmp/golden/ | ||
| for f in "${GOLDEN_FILES[@]}"; do | ||
| adb push "/tmp/golden/xnnpack/$f" /data/local/tmp/executorch/ | ||
| done |
There was a problem hiding this comment.
This script defines ADB_PATH=$ANDROID_HOME/platform-tools/adb but the newly added commands use plain adb instead of $ADB_PATH. If adb isn’t on PATH (or points to a different version), the script can fail or behave inconsistently. Use $ADB_PATH for the added shell/push commands for consistency with the earlier lines.
Use curl -f so HTTP errors (403/404) fail the script instead of silently saving an HTML error page. Unify all adb invocations to use $ADB_PATH for consistency with the rest of the script.
Check for large test assets on the connected device after setup and print a warning listing any missing files, pointing developers to scripts/run_android_emulator.sh for download URLs and adb push commands.
Summary
vit_b_16.pte (~330MB) and stories.pte (~29MB) are no longer bundled in the androidTest APK. Instead they are uploaded as a separate CI artifact and pushed to /data/local/tmp/executorch/ via adb before tests run. TestFileUtils.prepareTestFile() checks the push directory first and falls back to APK resources, so both CI and local workflows remain functional. This reduces the APK from ~361MB to ~2MB.
Test plan
CI