Android ROM device support and bringup tool, designed for maximum automation and speed.

Overview

adevtool

Android ROM device support and bringup tool, designed for maximum automation and speed.

Demo video on asciinema

Features

This tool automates the following tasks for devices that mostly run AOSP out-of-the-box (e.g. Google Pixel):

  • Downloading factory images and full OTA packages
  • Generating a list of proprietary files
  • Resolving overridden build rules and building modules from source (when possible)
  • Extracting, converting, and mounting factory images (supported source formats)
  • Extracting proprietary files
  • Extracting bootloader and radio firmware
  • Finding and adding missing system properties
  • Overriding build fingerprint to help pass SafetyNet
  • Adding missing SELinux policies
  • Adding missing HALs to vendor interface manifests
  • Generating resource overlays for device configs
  • Fixing privileged app signing certificates referenced in SELinux policies

This typically results in better device support with fewer bugs and issues, and makes it possible to quickly add support for new devices.

Example generated vendor modules for Pixel devices

Non-AOSP devices

Pixel devices will benefit from the most automation, but several features can still be used to ease manual bringup on other devices:

  • Extract files from proprietary-files.txt up to 2000% faster than LineageOS extract-utils (speed comparison)
    • LineageOS extract-utils: 1 min 27 sec
    • adevtool: 4 sec
    • Tested with Pixel 5 (redfin), cache cleared before testing
  • Compare a built system against stock images to find missing files, properties, and vendor interface declarations
  • List system files relevant to bringup (in all partitions)
  • Resolve overridden build rules from build warnings
  • Check SELinux policies to identify apps that should be presigned

Installation

Pre-requisites:

  • Node.js
  • Optional:
    • For converting sparse factory images: simg2img
    • For extracting OTA packages: payload-dumper-go
    • For extracting OTAs or factory images: unzip
    • For generating overlays: AAPT2 (included in AOSP and Android SDK)

To run adevtool without installation:

npx adevtool

To install adevtool globally using npm:

npm install -g adevtool

Using Yarn:

yarn global add adevtool

Usage

Why TypeScript?

TypeScript is not a common language for Android-related tools, but I picked it for several reasons:

  • Fast (enough) and easily parallelizable
  • Good static type system (for developer experience: fewer bugs and better editor code intelligence)
  • Good libraries for friendly CLI interfaces with pretty output
  • Relatively lightweight

Why not...

  • Python: May not meet speed/parallelization goals easily, types aren't as nice
  • Kotlin: Requires big+heavy JVM to run, CLI libraries aren't as good
  • Go: Good choice overall, but error checking can be overly verbose

Overall, TypeScript is a decent compromise on all of these points.

License

This tool is licensed under the MIT license, ensuring that anyone is free to use it for any purpose in compliance with the license. Contributions are welcome!

Comments
  • Generate-prep/all throws various errors depending by node version

    Generate-prep/all throws various errors depending by node version

    Hello there, I'm running Ubuntu 20.04 with npm installed through nvm-sh. While using generate-prep (or generate-all), the error vary depending on node version:

    Command used: adevtool generate-prep -s ~/dump/denniz_a16/ -b rp1a.200720.011 denniz.yaml

    Dump used: https://gitlab.com/Giovix92/denniz_dump

    Tinkered a bit with nvm, and found out the errors are different depending by node version:

    • Node v12.22.0 (suggested): SyntaxError: Unexpected token '?'
    • Node v16.13.2 (lts/Gallium): ReferenceError: structuredClone is not defined
    • Node v17.x.0: [...] Extracting properties ⠋ TypeError: Cannot read properties of undefined (reading 'split')

    YAML used:

    device:
      name: denniz
      vendor: oneplus
    
    platform:
      product_makefile: device/oneplus/denniz/lineage_denniz.mk
    
      namespaces:
        - hardware/oplus
    
    generate:
      files: true
      props: true
      overlays: true
      vintf: true
    

    Does that depend by my system config, my dump or by node/node package itself?

    Thank you in advance!

    opened by Giovix92 7
  • adevtool broken when extracting files from /system on Android 13

    adevtool broken when extracting files from /system on Android 13

    Extracting stock system source   sysroot/coral-tpp2.220218.010
    Enumerating files   vendor
    Replacing blobs with buildable modules
    Marking apps as presigned ⠴ app/Traceur/Traceur.apkError: Command failed: out/host/linux-x86/bin/aapt2 dump packagename /tmp/adevtool-50360044/sysroot/coral-tpp2.220218.010/app/Traceur/Traceur.apk
    ziparchive W 04-21 17:04:44 379225 379225 Unable to open '/tmp/adevtool-50360044/sysroot/coral-tpp2.220218.010/app/Traceur/Traceur.apk': No such file or directory
    /tmp/adevtool-50360044/sysroot/coral-tpp2.220218.010/app/Traceur/Traceur.apk: error: failed opening zip: I/O error.
    
        at ChildProcess.exithandler (node:child_process:398:12)
        at ChildProcess.emit (node:events:527:28)
        at maybeClose (node:internal/child_process:1090:16)
        at Socket.<anonymous> (node:internal/child_process:449:11)
        at Socket.emit (node:events:527:28)
        at Pipe.<anonymous> (node:net:715:12)
    

    Seems to be some sort of path issue where it incorrectly skips system/system instead of just system?

    opened by flawedworld 2
  • Throws Error: No supported source format found in <dir> when generate-prep command is executed

    Throws Error: No supported source format found in when generate-prep command is executed

    I was trying to build ROM for Pixel 4a(sunfish). I downloaded the vendor package using

    adevtool download stock_images -d sunfish it successfully downloads the package

    but when I execute adevtool generate-prep -s stock_images -b sq1d.220205.002 sunfish.yml I get the following error

    Extracting stock system source ⠋ Error: No supported source format found in 'stock_images'

    How can this be solved?

    opened by naive-guy 2
  • Add support for using a local archive of factory and OTA images

    Add support for using a local archive of factory and OTA images

    Many individuals working full time on device development often maintain a local archive of releases, and it would be useful if there was an easier usage of such a scenario.

    opened by flawedworld 2
  • adevtool violates PRODUCT_ENFORCE_ARTIFACT_PATH_REQUIREMENTS on redfin

    adevtool violates PRODUCT_ENFORCE_ARTIFACT_PATH_REQUIREMENTS on redfin

    With the configuration in https://github.com/flawedworld/adevtool/tree/adevtool-mainline-issue adevtool violates mainline checks on redfin.

    [ 99% 260/261] finishing build rules ...
    FAILED:
    build/make/core/artifact_path_requirements.mk:26: warning:  vendor/google_devices/redfin/redfin.mk produces files inside build/make/target/product/generic_system.mks artifact path requirement.
    Offending entries:
    system/lib64/[email protected]
    In file included from build/make/core/main.mk:1345:
    build/make/core/artifact_path_requirements.mk:26: error: Build failed.
    22:44:52 ckati failed with: exit status 1
    
    opened by flawedworld 2
  • How to generate a device-specific YAML config file

    How to generate a device-specific YAML config file

    I have a rockchip rk3399-based device with its AOSP 10 BSP. I am trying to update it to AOSP 11. I am hoping to use https://github.com/kdrag0n/adevtool/blob/main/docs/pixel-generate.md but cannot figure out a way to obtain or generate my device's YAML config file. Any more information on this topic would be very helpful. Thanks!

    opened by keyur2maru 1
  • build: Fix duplicate generated modules for cross-partition conflicts

    build: Fix duplicate generated modules for cross-partition conflicts

    The partition check for conflicts fails when we have cross-partition conflicts, as the same-partition module may also be a conflict.

    To fix the issue, keep a separate map of all candidate modules with the same name and check all of them for duplicate entries, even if some fail the partition check.

    opened by kdrag0n 0
  • SyntaxError: Unexpected token r in JSON at position 1

    SyntaxError: Unexpected token r in JSON at position 1

    keyurmaru@gandalf:~/rk3399/rk3399c_android10.0_tesa$ adevtool collect-state out/hello/ -d rk3399_firefly_aioc_ai_lvds
    Enumerating files   odm_dlkm
    Extracting properties   
    Extracting SELinux contexts   
    Extracting overlays ⠼ /home/keyurmaru/rk3399/rk3399c_android10.0_tesa/out/target/product/rk3399_firefly_aioc_ai_lvds/vendor/overlay/framework-res__auto_generated_rro_vendor.apk    SyntaxError: Unexpected token r in JSON at position 1
    
    
    opened by keyur2maru 1
  • please make it work.

    please make it work.

    Hi I'm comming fr here:

    https://github.com/bkerler/oppo_decrypt/issues/75#issuecomment-1317001111

    My device tree and kernel are here:

    https://bgzashtita.es/git/Android-smartphones/realme/realme-5pro/peturbg/device-tree-13

    https://bgzashtita.es/git/Android-smartphones/realme/realme-5pro/peturbg/kernel

    If you see branch you will see that I'm making experiments try make work like stockrm. And can't do it. I try compile latest version of cherish 4.2 and it work but slow. Unknown what problema has. It comment about missing thing. Click frequency. Om debice tree there is logs foles from staet to finish. Maybe this developer use confirmation for anddldi 12 and android 13 need to be updated or removed. Please someone help me I don't know what to do. Here is extracted content https://github.com/noobyysauraj/ofp_to_zip/issues/2#issuecomment-1317038372 it is f06 this is extracted .ofp becouse I don't find .ofp to .zip programa is there way make this folders to be used by adevtool.

    opened by bgman111111 0
  • Error during AOSP build after adding adevtool-extracted blobs

    Error during AOSP build after adding adevtool-extracted blobs

    Hello there,

    I'm trying to compile AOSP branch android-12.1.0_r7 for sunfish (Pixel 4a) device.

    After using adevtool to extract the blobs (and put under vendor/google_devices) I encounter errors about sepolicy during the build, that stop it.

    Here they are:

    [ 51% 75020/144890] build out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy
    FAILED: out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy
    /bin/bash -c "(ASAN_OPTIONS=detect_leaks=0 out/host/linux-x86/bin/checkpolicy -M -c             30 -o out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.tmp out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.recovery.conf ) && (out/host/linux-x86/bin/sepolicy-analyze out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.tmp permissive > out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.permissivedomains ) && (if [ \"eng\" = \"user\" -a -s out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.permissivedomains ]; then            echo \"==========\" 1>&2;               echo \"ERROR: permissive domains not allowed in user builds\" 1>&2;           echo \"List of invalid domains:\" 1>&2;              cat out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.permissivedomains 1>&2;          exit 1;   
                fi ) && (mv out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy.tmp out/target/product/sunfish/obj/ETC/sepolicy.recovery_intermediates/sepolicy )"
    device/google/sunfish-sepolicy/vendor/qcom/common/attributes:2:ERROR 'Duplicate declaration of type' at token ';' on line 6386:
    attribute vendor_persist_type;
    #line 1 "device/google/sunfish-sepolicy/vendor/qcom/common/attributes"
    checkpolicy:  error(s) encountered while parsing configuration
    
    [ 51% 75021/144890] build out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows
    FAILED: out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows
    /bin/bash -c "(ASAN_OPTIONS=detect_leaks=0 out/host/linux-x86/bin/checkpolicy -M -c             30 -o out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows.tmp out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/policy.conf ) && (out/host/linux-x86/bin/sepolicy-analyze out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows.tmp neverallow -w -f out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/policy_2.conf ||           ( echo \"\" 1>&2;         echo \"sepolicy-analyze failed. This is most likely due to the use\" 1>&2;            echo \"of an expanded attribute in a neverallow assertion. Please fix\" 1>&2;         echo \"the policy.\" 1>&2;      exit 1 ) ) && (touch out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows.tmp ) && (mv out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows.tmp out/target/product/sunfish/obj/FAKE/sepolicy_neverallows_intermediates/sepolicy_neverallows )"
    device/google/sunfish-sepolicy/vendor/qcom/common/attributes:2:ERROR 'Duplicate declaration of type' at token ';' on line 6386:
    attribute vendor_persist_type;
    #line 1 "device/google/sunfish-sepolicy/vendor/qcom/common/attributes"
    checkpolicy:  error(s) encountered while parsing configuration
    [ 51% 75068/144890] //frameworks/base:framework-cppstream-protos generate 2
    frameworks/base/core/proto/android/os/batteryusagestats.proto:22:1: warning: Import frameworks/proto_logging/stats/enums/os/enums.proto but not used.
    17:15:28 ninja failed with: exit status 1
    

    Do you have an idea of what could be the culprit?

    Kind regards

    opened by ConradGyllenhaal 1
  • asciinema recording archived in README.md

    asciinema recording archived in README.md

    The linked asciinema recording at the top of the README is archived and not working. I have a screenshot below.

    Screenshot of archived asciinema recording in README.md

    Clicking on the asciinema recording directs to the "This recording has been archived" page, shown in the screenshot below.

    asciinema page with the message, "This recording has been archived / All unclaimed recordings (the ones not linked to any user account) are automatically archived 7 days after upload."

    opened by taivlam 0
Owner
Danny Lin
Android & web developer
Danny Lin
Boilerplate project to run MOBILE Test Automation with WebdriverIO v7, Mocha, Appium, Allure reporting and Momentum Suite cloud device farm support

WebdriverIO Mocha Appium Momentumsuite WebdriverIO Integration with local or Momentum Suite real mobile farm devices Supports Native or Hybrid Android

Momentum Suite 21 Dec 5, 2022
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
A type speed checking website which lets you check your typing speed and shows the real-tme leaderboards with mongodb as DB and express as backend

This is a Next.js project bootstrapped with create-next-app. Getting Started First, run the development server: npm run dev # or yarn dev Open http://

Sreehari jayaraj 8 Mar 27, 2022
A new generation GUI automation framework for Web and Desktop Application Testing and Automation.

Clicknium-docs Clicknium is a new generation GUI automation framework for all types of applications. It provides easy and smooth developer experience

null 109 Dec 19, 2022
Custom touch bar or digital macropad app for GNU/Linux 🐧 using an Android device 📱

?? Boar ??️ Custom touch bar or digital macropad app for GNU/Linux ?? using an Android device ?? Those apps allow creating a keyboard with custom keys

Fabián Velosa 0 Oct 29, 2022
Device.js is a JavaScript library to detect device, viewport, and browser information using plain JavaScript.

Device.js Device.js is a JavaScript library to detect device, viewport, and browser information using plain JavaScript. Compatibility Works with all m

Emanuel R. Vásquez 5 Dec 16, 2022
Recompo is a free and open-source project that is designed for customizability and speed and is easy to use and powerful.

About Recompo is a free and open-source project that is designed for customizability and speed and is easy to use and powerful Installation recompo is

recompo 4 Mar 20, 2022
Open source data infrastructure platform. Designed for developers, built for speed.

Gigahex is a web based data infrastructure platform to deploy and manage Apache Spark™, Apache Kafka and Apache Hadoop clusters. Currently, it support

Gigahex 22 Dec 6, 2022
Open source data infrastructure platform. Designed for developers, built for speed.

Gigahex is a web based data infrastructure platform to deploy and manage Apache Spark™, Apache Kafka and Apache Hadoop clusters. Currently, it support

Gigahex 21 Apr 1, 2022
Calculates maximum composite SLA for a list of sequentially provided cloud services or your custom-defined services.

SlaMax Calculates maximum composite SLA for a list of sequentially provided cloud services or your custom-defined services. Here are a few use-cases y

Mikael Vesavuori 4 Sep 19, 2022
A server reimplementation for Wangan Midnight Maximum Tune 6.

Bayshore Wangan Midnight Maximum Tune 6 server reimplementation written in TypeScript Credits This software is part of Project Asakura. At this time,

Project朝倉 20 Dec 23, 2022
This plugin integrates by default with Twitter bootstrap using badges to display the maximum lenght of the field where the user is inserting text. Uses the HTML5 attribute "maxlength" to work.

Bootstrap MaxLength This plugin integrates by default with Twitter bootstrap using badges to display the maximum length of the field where the user is

Maurizio 772 Dec 25, 2022
🍉 Water is a micro-ORM + QueryBuilder designed to facilitate queries and operations on PostgreSQL databases designed to work in Melon

?? Water Water is a micro-ORM + QueryBuilder designed to facilitate queries and operations on PostgreSQL databases designed to work in MelonRuntime In

Melon Runtime 22 Aug 6, 2022
Query for CSS brower support data, combined from caniuse and MDN, including version support started and global support percentages.

css-browser-support Query for CSS browser support data, combined from caniuse and MDN, including version support started and global support percentage

Stephanie Eckles 65 Nov 2, 2022
A minimal yet powerful HTTP client/API testing tool made for speed.

req req is a lightweight, minimal yet powerful HTTP client slash API testing tool designed for speed. Contents Features Installation Documentation Con

Siddharth 37 Aug 29, 2022
Minimalistic, opinionated, and predictable release automation tool.

Release Minimalistic, opinionated, and predictable release automation tool. General idea Think Prettier but for automated releases: minimalistic, opin

Open Source: JavaScript 173 Dec 18, 2022
An unofficial companion tool created for use alongside PhotoPrism to enable API endpoints and automation.

PhotoPrism Helper PhotoPrism Helper is an unofficial companion tool created for use alongside PhotoPrism. This project isn't associated with the Photo

Ryan Miller 9 Dec 25, 2022