Android dm-verity 实现原理深入研究

前言:这篇文章是 Kevin 的原创作品,如有相关疑问,可以留言交流。

思维导图:

dm-verify
dm-verity

 

 

说明:源码基于 SC20 平台 Android5.1
Android dm-verify overview

目录

Android dm-verify overview.. 1

一、原理… 1

与Verified Boot关系… 1

dm-verity. 1

作用分区… 2

二、模块结构… 2

1.签名… 2

生成OEM自己的密钥对… 4

验签… 5
用户空间,android 部分… 5

内核空间… 5

三、如何启用… 5

四、测试… 6

测试样例1. 无法 remount, 无法 push 文件… 6

测试样例2. 6

五、存在风险… 6

物理块出现坏块… 6

六、其他… 6

七、参考文档… 6

一、原理

Verified Boot关系

Verified Boot 是 Android 4.4 开始引入的一个新特性,配合可选的 dm-verify 功能,可以检测系统是否被篡改,以此保存系统的完整性。

dm-verity

dm-verity 基于kernel 的 Device mapper 框架,Device mapper 是 Linux 2.6 内核中提供的一种从逻辑设备到物理设备的映射框架机制,在该机制下,用户可以很方便的根据自己的需要制定实现存储资源的管理策略。关于 Device mapper,可以参考此文献

https://www.ibm.com/developerworks/cn/linux/l-devmapper/

dm-verity 用一个 hash 树来描述整个 system 镜像。这种机制允许 system 分区在读写的时候进行校验,而不是一次性将整个 system 镜像进行校验。当校验 hash 值不一致的时候,返回 IO 错误

框架示意图:

dm-verity-hash-tree
dm-verity-hash-tree

 作用分区

system

vendor

OEM

其他只读分区

二、模块结构

 1.签名

如何生成用于dm-verity 校验的镜像,可以参考一下主流程:

主流程,Android 官方文档如下:

– Generate an ext4 system image.

– Generate a hash tree for that image.

– Build a dm-verity table for that hash tree.

– Sign that dm-verity table to produce a table signature.

– Bundle the table signature and dm-verity table into verity metadata.

– Concatenate the system image, the verity metadata, and the hash tree.

  • 生成 ext4 格式的 system 镜像
  • 生成 system 镜像的 hash 树
  • 根据 hash 树生成 dm-verity table
  • 对 dm-verity 进行签名,得到签名文件
  • 将签名、dm-verity table 打包到 metadata 镜像
  • 将 verity metadata,hash 树 添加到 system 镜像末尾
  • 生成 ext4 格式的 system 镜像
    build/tools/releasetools/build_image.py  BuildImage 函数 中

通过 prop_dict 属性,判断是否启用 verity 功能,启用的话,调整 system 分区大小,以便后面添加相关文件到 system.img 末尾,然后 RunCommand(build_command) 生成初始的 system.img。

prop_dict 文件路径:

out/target/product/msm8909/obj/PACKAGING/systemimage_intermediates/system_image_info.txt

prop_dict 文件内容:

fs_type=ext4

system_size=1288491008

userdata_size=4831838208

cache_fs_type=ext4

cache_size=268435456

extfs_sparse_flag=-s

selinux_fc=out/target/product/msm8909/root/file_contexts

verity=true

verity_key=build/target/product/security/verity

verity_signer_cmd=out/host/linux-x86/bin/verity_signer

system_verity_block_device=/dev/block/bootdevice/by-name/system

skip_fsck=true

  • 生成 system 镜像的 hash

BuildVerityTree 函数生成 hash tree (verity_image)

步骤3,4,5BuildVerityMetadata中实现,BuildVerityMetadata 调用 build_verity_metadata.py 来实现

BuildVerityMetadata 生成 metadata (metadata_image),

跳转到 system/extras/verity/build_verity_metadata.py

  • 根据 hash 树生成 dm-verity table

build_verity_table 生成 dm-table

dm-table 其实就是一个字符串

  • dm-verity 进行签名,得到签名文件

sign_verity_table 将 dm-table 签名

signer_key = build/target/product/security/verity.pk8

  • 将签名、dm-verity table 打包到 metadata 镜像

build_metadata_block 将 dm-table 和 签名信息打包,写入 datameta.img

  • verity metadatahash 添加到 system 镜像末尾

build/tools/releasetools/build_image.py  BuildVerifiedImage 生成最终的可以用于 dm-verity 校验的镜像

Append2Simg 添加 verity_image 和 datameta.img 到 system.img 末尾

其他

FIXED_SALT 可以修改为自己的 salt

如何生成自己的oem key

dm-verity 相关的密钥

build/target/product/security/ build/target/product/verity.mk verity.pk8 – 私钥,用于签名 boot.img 和 system.img

verity.x509.pem – 包含公钥的证书

verity_key – 公钥,dm verity 中用于验签 system 分区

   生成OEM自己的密钥对

  1. HSM(Hardware Security Module)
  1. OpenSSL tool
  • 生成新得密钥对 >openssl version OpenSSL 1.0.2d 9 Jul 2015 >development/tools/make_key mykey ‘/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com’
  • 为DM-Verity 功能生成 verity key
    1. 使用下面的命令来生成 verity key 的工具 generate_verity_key: source build/envsetup.sh choosecombo make generate_verity_key (mmm system/extras/verity/)
    2. 将 *.x509.pem 转换成 verity key generate_verity_key 的代码位于:system/extra/verity/generate_verity_key.c generate_verity_key 的用法:generate_verity_key | -convert 实例: out/host/linux-x86/bin/generate_verity_key -convert mykey.x509.pem verity_key
  • 拷贝并重命名
    1. 拷贝pk8,mykey.x509.pem,verity_key.pub 至 build/target/product/security/ 目录,将其重命名: verity.pk8, verity.x509.pem,verity_key ,并替换默认的开发 key。
  • 生成 keystore
    1. 执行下面的两个命令,生成img openssl rsa -in mykey.pk8 -inform DER -pubout -outform DER -out mypub.der java -Xmx512M -jar out/host/linux-x86/framework/KeystoreSigner.jar mykey.pk8 mykey.x509.pem keystore.img mypub.der
    2. 通过下面的脚本将img生成oem_keystore.h文件,shell 输入: function generate_oem_keystore_h() { echo \#ifndef __OEM_KEYSTORE_H echo \#define __OEM_KEYSTORE_H xxd -i $1 | sed -e ‘s/unsigned char .* = {/const unsigned char OEM_KEYSTORE[] = {/g’ -e ‘s/unsigned int .* =.*;//g’ echo \#endif }
  • generate_oem_keystore_h keystore.img > oem_keystore.h
  • 将该h文件拷贝到:bootable/bootloader/lk/platform/msm_shared/include
  1. 验签

用户空间,android 部分

相关文件:

system/core/fs_mgr/fs_mgr_verity.c

用户空间对 dm-verity 进行初始化,验签 hash_table 的签名,传入 hash_table 等参数

内核空间

相关文件:

KERNEL_SRC/driver/md/dm-verity.c 相关代码

内核空间根据用户空间传入的参数,进行初始化

当有 IO 操作时,对相应 data block 进行验签。

data block 读取时,不仅当前 data block 需要 hash 校验,上一层的 hash block 也需要进行校验,直到root hash。

每一个 data block 如果已经验签过,再次读取时就不用进行层层的校验,只校验当前 data block 的 hash 是否正确即可。

三、如何启用

  1. kernel 配置文件使能 CONFIG_DM_VERITY

kernel/arch/arm/configs/msm8909-1gb_defconfig

  1. Android 相关 mk 文件配置

device/qcom/msm8909/msm8909.mk PRODUCT_SUPPORTS_VERITY := true PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/bootdevice/by-name/system

  1. 更新 fstab,system 分区添加 verify 标志

/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait,verify

  1. 编译 userdebug 或者 user 版本 boot.img 以及 system.img,烧录

四、测试

测试样例1. 无法 remount, 无法 push 文件

Mount | grep system /dev/block/dm-0 /system ext4 ro, seclable, relatime, data=ordered 0 0

adb pull /fstab.qcom c:\temp\fstab, 检查 verify 标志: /dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait, verify

adb remount Use “adb disable-verity” to disable verity. If you do not, remount may succeed, however, you will still not be able to write to these volumes. remount of system failed: Read-only file system remount failed

 测试样例2.

(1).先烧录启用 verity 功能的 boot.img 和 system.img

(2). 然后烧录未启用 verity 功能的 boot.img,重启后 push 一个 apk 到 /system/app/ 目录

(3). 重新烧录启用了 verity 功能的 boot.img,重启后,机器无法开机,kernel log 可以看到 data block xxx is corrupted

五、存在风险

 物理块出现坏块

  1. 开机检验出错,无法开机
  2. 开机检验没问题,读取文件出错,返回 Error IO

六、其他

软件集成和 OTA 升级,必须使用 block 方式生成 OTA 升级包,需要注意适配。

 七、参考文档

Verified Boot

https://source.android.com/security/verifiedboot/index.html

Verifying Boot

https://source.android.com/security/verifiedboot/verified-boot.html

Implementing dm-verity

https://source.android.com/security/verifiedboot/dm-verity.html

Dmverity

https://gitlab.com/cryptsetup/cryptsetup/wikis/DMVerity

dm-table format

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/device-mapper/verity.txt

Device mapper

https://www.ibm.com/developerworks/cn/linux/l-devmapper/


网友遇到的问题1:

问题现象:
fs_mgr: Failed to get verity metadata ‘/dev/block/bootdevice/by-name/system’ (Invalid argument)

解决方案:

博主 Kevin 也遇到这个问题了。这个问题在6.0以上的 Android 上可能会遇到。主要是实际 partition size 和 BoardConfig.mk 中的 system image size 的两个数值的大小不匹配,而且数值必须被4K整除。
解决方法:
1. 修改相关的分区表大小,以及 BoardConfig.mk 中 BOARD_SYSTEMIMAGE_PARTITION_SIZE 为相同的数值,并且必须被4K整除。
2. 如果还是有问题,那还要修改 system/extras/libfec/fec_open.cpp
int fec_open(struct fec_handle **handle, const char *path, int mode, int flags,
f->data_size = f->size; /* until ecc and/or verity are loaded */, change this to
f->data_size = 1610612736; /* until ecc and/or verity are loaded */

1610612736修改为具体项目的数值,如此这般,问题应该就可以解决了。

《Android dm-verity 实现原理深入研究》上有14条评论

      1. 请教博主个问题,我现在按照你的启用步骤打开了verity,但是系统现在启动到init进程的起不来了。 可以帮忙分析下什么原因不? log 如下:
        fs_mgr: Failed to get verity metadata ‘/dev/block/platform/fe330000.sdhci/by-name/system’ (No such file or directory)
        [ 2.401986] fs_mgr: Could not set up verified partition, skipping!
        [ 2.402054] fs_mgr: check_fs(): mount(/dev/block/platform/fe330000.sdhci/by-name/cache,/cache,ext4)=-1: No such file or directory
        [ 2.402076] fs_mgr: Not running /system/bin/e2fsck on /dev/block/platform/fe330000.sdhci/by-name/cache (executable not in system image)
        [ 2.414721] EXT4-fs (mmcblk1p9): recovery complete
        [ 2.415166] EXT4-fs (mmcblk1p9): mounted filesystem with ordered data mode. Opts: noauto_da_alloc,discard
        [ 2.415268] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/cache,target=/cache,type=ext4)=0
        [ 2.415644] EXT4-fs (mmcblk1p11): Ignoring removed nomblk_io_submit option
        [ 2.416543] EXT4-fs (mmcblk1p11): warning: maximal mount count reached, running e2fsck is recommended
        [ 2.417003] EXT4-fs (mmcblk1p11): recovery complete
        [ 2.417012] EXT4-fs (mmcblk1p11): mounted filesystem with ordered data mode. Opts: errors=remount-ro,nomblk_io_submit
        [ 2.417046] fs_mgr: check_fs(): mount(/dev/block/platform/fe330000.sdhci/by-name/metadata,/metadata,ext4)=0: Success
        [ 2.433835] fs_mgr: check_fs(): unmount(/metadata) succeeded
        [ 2.433880] fs_mgr: Not running /system/bin/e2fsck on /dev/block/platform/fe330000.sdhci/by-name/metadata (executable not in system image)
        [ 2.435099] EXT4-fs (mmcblk1p11): warning: maximal mount count reached, running e2fsck is recommended
        [ 2.435633] EXT4-fs (mmcblk1p11): mounted filesystem with ordered data mode. Opts: noauto_da_alloc,discard
        [ 2.435678] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/metadata,target=/metadata,type=ext4)=0
        [ 2.435739] fs_mgr: Running /system/bin/fsck.f2fs -a /dev/block/platform/fe330000.sdhci/by-name/userdata
        [ 2.436788] fsck.f2fs: executing /system/bin/fsck.f2fs failed: No such file or directory
        [ 2.436788]
        [ 2.436817] fsck.f2fs: fsck.f2fs terminated by exit(255)
        [ 2.436817]
        [ 2.514452] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/userdata,target=/data,type=f2fs)=0
        [ 2.516245] init: (Loading properties from /system/build.prop took 0.00s.)
        [ 2.516293] init: (Loading properties from /vendor/build.prop took 0.00s.)
        [ 2.516322] init: (Loading properties from /factory/factory.prop took 0.00s.)
        [ 2.516529] init: /recovery not specified in fstab
        [ 2.516595] init: Starting service ‘healthd’…
        [ 2.517042] init: cannot find ‘/system/vendor/bin/pvrsrvctl’ (No such file or directory), disabling ‘pvrsrvctl’
        [ 2.517074] init: cannot find ‘/system/bin/displayd’ (No such file or directory), disabling ‘displayd’
        [ 2.517111] init: cannot find ‘/system/vendor/bin/elogs.sh’ (No such file or directory), disabling ‘earlylogs’
        [ 2.517147] init: cannot find ‘/system/bin/app_process64’ (No such file or directory), disabling ‘zygote’
        [ 2.517177] init: cannot find ‘/system/bin/app_process32’ (No such file or directory), disabling ‘zygote_secondary’
        [ 2.519257] binder: 191:191 transaction failed 29189, size 0-0
        [ 2.520217] init: insmod: open(“/system/lib/modules/ump.ko”) failed: No such file or directoryinit: insmod: open(“/system/lib/modules/mali.y
        [ 2.521509] zram0: detected capacity change from 0 to 533413888
        [ 2.523097] Unable to find swap-space signature
        [ 2.523215] fs_mgr: swapon failed for /dev/block/zram0
        [ 2.523673] init: do_start: Service debuggerd not found
        [ 2.523693] init: do_start: Service debuggerd64 not found
        [ 2.523710] init: do_start: Service vold not found
        [ 2.524740] init: Not bootcharting.
        [ 2.560286] init: cannot find ‘/system/bin/tzdatacheck’ (No such file or directory), disabling ‘exec 1 (/system/bin/tzdatacheck)’
        [ 2.563376] audit: type=1400 audit(1358534124.043:3): avc: denied { setattr } for pid=1 comm=”init” name=”cifsmanager” dev=”mmcblk1p13″ 1
        [ 2.579632] F2FS-fs (mmcblk1p13): acl options not supported
        [ 2.579665] F2FS-fs (mmcblk1p13): Unrecognized mount option “errors=panic” or missing value
        [ 2.584165] init: write_file: Unable to open ‘/proc/sys/kernel/core_pattern’: No such file or directory
        [ 2.584325] init: (Loading properties from /data/local.prop took 0.00s.)
        [ 2.584560] init: do_start: Service logd not found
        [ 2.584578] init: do_start: Service logd-reinit not found
        [ 2.585272] init: write_file: Unable to open ‘/proc/sys/vm/min_free_order_shift’: No such file or directory
        [ 2.586928] init: SELinux: Could not stat /sys/devices/system/cpu/cpufreq/interactive: No such file or directory.
        [ 2.588051] init: write_file: Unable to open ‘/proc/sys/vm/lazy_vfree_tlb_flush_all_threshold’: No such file or directory
        [ 2.591053] init: cannot find ‘/system/xbin/vm’ (No such file or directory), disabling ‘vm_daemon’
        [ 2.592748] init: property ‘ro.serialno’ doesn’t exist while expanding ‘${ro.serialno}’
        [ 2.592787] init: write: cannot expand ‘${ro.serialno}’
        [ 2.592813] init: property ‘ro.product.manufacturer’ doesn’t exist while expanding ‘${ro.product.manufacturer}’
        [ 2.592829] init: write: cannot expand ‘${ro.product.manufacturer}’
        [ 2.592856] init: property ‘ro.product.model’ doesn’t exist while expanding ‘${ro.product.model}’
        [ 2.592873] init: write: cannot expand ‘${ro.product.model}’
        [ 2.593471] file system registered
        [ 2.594175] using random self ethernet address
        [ 2.594188] using random host ethernet address
        [ 2.594251] init: write_file: Unable to open ‘/config/usb_gadget/g1/functions/rndis.gs4/wceis’: Permission denied
        [ 2.601401] init: cannot find ‘/system/bin/sh’ (No such file or directory), disabling ‘console’
        [ 2.601524] init: Starting service ‘adbd’…
        [ 2.602305] init: cannot find ‘/system/bin/update_verifier’ (No such file or directory), disabling ‘exec 2 (/system/bin/update_verifier)’
        [ 2.602342] init: cannot find ‘/system/bin/rild’ (No such file or directory), disabling ‘ril-daemon’
        [ 2.602365] init: cannot find ‘/system/bin/install-recovery.sh’ (No such file or directory), disabling ‘flash_recovery’
        [ 2.602395] init: cannot find ‘/system/bin/drmservice’ (No such file or directory), disabling ‘drmservice’
        [ 2.602418] init: cannot find ‘/system/bin/bplus_helper’ (No such file or directory), disabling ‘bplus_helper’
        [ 2.602439] init: cannot find ‘/system/bin/busybox’ (No such file or directory), disabling ‘up_eth0’
        [ 2.602475] init: cannot find ‘/system/bin/akmd’ (No such file or directory), disabling ‘akmd’
        [ 2.602515] init: cannot find ‘/system/bin/mediaserver’ (No such file or directory), disabling ‘media’
        [ 2.602542] init: cannot find ‘/system/vendor/bin/start_log_srv.sh’ (No such file or directory), disabling ‘ap_log_srv’
        [ 2.602564] init: cannot find ‘/system/vendor/bin/crashlogd’ (No such file or directory), disabling ‘crashlogd’
        [ 2.602591] init: cannot find ‘/vendor/bin/log-watch’ (No such file or directory), disabling ‘log-watch’
        [ 2.603471] init: write_file: Unable to open ‘/sys/class/android_usb/android0/enable’: Permission denied
        [ 2.603531] init: write_file: Unable to open ‘/sys/class/android_usb/android0/idVendor’: Permission denied
        [ 2.603587] init: write_file: Unable to open ‘/sys/class/android_usb/android0/idProduct’: Permission denied
        [ 2.603638] init: write_file: Unable to open ‘/sys/class/android_usb/android0/functions’: Permission denied
        [ 2.603683] init: write_file: Unable to open ‘/sys/class/android_usb/android0/enable’: Permission denied
        [ 3.325195] mmc2: error -110 whilst initialising SDIO card
        [ 3.338777] mmc_host mmc2: Bus speed (slot 0) = 300000Hz (slot req 300000Hz, actual 300000HZ div = 0)
        [ 3.519480] binder: 191:191 transaction failed 29189, size 0-0
        [ 4.071991] phy phy-ff770000.syscon:usb2-phy@e450.6: charger = USB_SDP_CHARGER
        [ 4.174370] fusb302 4-0022: PD disabled
        [ 4.176531] cdn-dp fec00000.dp: [drm:cdn_dp_pd_event_work] Not connected. Disabling cdn
        [ 4.190663] rockchip-dwc3 usb@fe800000: USB peripheral connected

        1. 抱歉,回复晚了。不知道你的问题解决没有。
          fs_mgr: Failed to get verity metadata ‘/dev/block/platform/fe330000.sdhci/by-name/system’ (No such file or directory)
          这段log看起来是 dm-verify 生成的签名信息没有生成到 system.img 中,所以启动的时候没有找到 verify metadata。verity metadata 是镜像生成的时候附件到 system.img 尾部的,所以你需要确认一下你是否已经成功的启用了 dm-verity,并且重新 clean, 生成了正确的 system.img 。

  1. 博主我也遇到这个问题了。这个问题在6.0以上的 Android 上可能会遇到。主要是实际 partition size 和 BoardConfig.mk 中的 system image size 的两个数值的大小不匹配,而且数值必须被4K整除。
    解决方法:
    1. 修改相关的分区表大小,以及 BoardConfig.mk 中 BOARD_SYSTEMIMAGE_PARTITION_SIZE 为相同的数值,并且必须被4K整除。
    2. 如果还是有问题,那还要修改 system/extras/libfec/fec_open.cpp
    int fec_open(struct fec_handle **handle, const char *path, int mode, int flags,
    f->data_size = f->size; /* until ecc and/or verity are loaded */, change this to
    f->data_size = 1610612736; /* until ecc and/or verity are loaded */

    1610612736修改为具体项目的数值,如此这般,问题应该就可以解决了。如此这般,问题应该就可以解决了。

  2. 博主,您好:
    android7.1的是不是有些变化了?发现要生成keystone.img无法生成,这个是什么原因?
    Error: Unable to access jarfile out/host/linux-x86/framework/KeystoreSigner.jar

  3. 刚刚有个问题不知道怎么解决。
    然后去百度搜索。。。
    然后搜索到 Kevin 我自己的博客上有解决方案。。。
    还是我的原创文章。。。
    @_@

  4. 博主,您好:
    android7.1遇到一个问题,望能帮忙解惑:ioctl(fd, DM_TABLE_LOAD, io)中io参数个数的限制在10个或11个,这个是为什么,我看5.1和8.0都没有这个限制。
    在dm-verity.c中函数static int verity_ctr(struct dm_target *ti, unsigned argc, char **argv)
    { …
    if (argc 11) {
    ti->error = “Invalid argument count: 10-11 arguments required”;
    r = -EINVAL;
    goto bad;
    }
    }
    实际IO的参数是20个:
    fs_mgr: loading verity table: ‘1 /dev/block/bootdevice/by-name/system /dev/block/bootdevice/by-name/system 4096 4096 258044 258044 sha256 4b401d9f595c05c45e910692a4f0695445e951f1d27031d3acdf2ebfea156f0a aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7 9 use_fec_from_device /dev/block/bootdevice/by-name/system fec_start 260085 fec_blocks 260085 fec_roots 2 ignore_zero_blocks’

    1. 这个细节没有留意,具体的项目需求决定了研究的方向以及深入程度。最近工作较忙,稍后有空研究一下,或者你有答案了,欢迎再来给大家解答。

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据