前言:这篇文章是 Kevin 的原创作品,如有相关疑问,可以留言交流。
思维导图:
说明:源码基于 SC20 平台 Android5.1
Android dm-verify overview
目录
Android dm-verify overview.. 1
一、原理… 1
与Verified Boot关系… 1
dm-verity. 1
作用分区… 2
二、模块结构… 2
1.签名… 2
生成OEM自己的密钥对… 4
验签… 5
用户空间,android 部分… 5
内核空间… 5
三、如何启用… 5
四、测试… 6
测试样例1. 无法 remount, 无法 push 文件… 6
测试样例2. 6
五、存在风险… 6
物理块出现坏块… 6
六、其他… 6
七、参考文档… 6
一、原理
Verified Boot 是 Android 4.4 开始引入的一个新特性,配合可选的 dm-verify 功能,可以检测系统是否被篡改,以此保存系统的完整性。
dm-verity 基于kernel 的 Device mapper 框架,Device mapper 是 Linux 2.6 内核中提供的一种从逻辑设备到物理设备的映射框架机制,在该机制下,用户可以很方便的根据自己的需要制定实现存储资源的管理策略。关于 Device mapper,可以参考此文献
https://www.ibm.com/developerworks/cn/linux/l-devmapper/
dm-verity 用一个 hash 树来描述整个 system 镜像。这种机制允许 system 分区在读写的时候进行校验,而不是一次性将整个 system 镜像进行校验。当校验 hash 值不一致的时候,返回 IO 错误
框架示意图:
system
vendor
OEM
其他只读分区
二、模块结构
1.签名
如何生成用于dm-verity 校验的镜像,可以参考一下主流程:
主流程,Android 官方文档如下:
– Generate an ext4 system image.
– Generate a hash tree for that image.
– Build a dm-verity table for that hash tree.
– Sign that dm-verity table to produce a table signature.
– Bundle the table signature and dm-verity table into verity metadata.
– Concatenate the system image, the verity metadata, and the hash tree.
即
- 生成 ext4 格式的 system 镜像
- 生成 system 镜像的 hash 树
- 根据 hash 树生成 dm-verity table
- 对 dm-verity 进行签名,得到签名文件
- 将签名、dm-verity table 打包到 metadata 镜像
- 将 verity metadata,hash 树 添加到 system 镜像末尾
- 生成 ext4 格式的 system 镜像
build/tools/releasetools/build_image.py BuildImage 函数 中
通过 prop_dict 属性,判断是否启用 verity 功能,启用的话,调整 system 分区大小,以便后面添加相关文件到 system.img 末尾,然后 RunCommand(build_command) 生成初始的 system.img。
prop_dict 文件路径:
out/target/product/msm8909/obj/PACKAGING/systemimage_intermediates/system_image_info.txt
prop_dict 文件内容:
fs_type=ext4
system_size=1288491008
userdata_size=4831838208
cache_fs_type=ext4
cache_size=268435456
extfs_sparse_flag=-s
selinux_fc=out/target/product/msm8909/root/file_contexts
verity=true
verity_key=build/target/product/security/verity
verity_signer_cmd=out/host/linux-x86/bin/verity_signer
system_verity_block_device=/dev/block/bootdevice/by-name/system
skip_fsck=true
- 生成 system 镜像的 hash 树
BuildVerityTree 函数生成 hash tree (verity_image)
步骤3,4,5在BuildVerityMetadata中实现,BuildVerityMetadata 调用 build_verity_metadata.py 来实现
BuildVerityMetadata 生成 metadata (metadata_image),
跳转到 system/extras/verity/build_verity_metadata.py
- 根据 hash 树生成 dm-verity table
build_verity_table 生成 dm-table
dm-table 其实就是一个字符串
- 对 dm-verity 进行签名,得到签名文件
sign_verity_table 将 dm-table 签名
signer_key = build/target/product/security/verity.pk8
- 将签名、dm-verity table 打包到 metadata 镜像
build_metadata_block 将 dm-table 和 签名信息打包,写入 datameta.img
- 将 verity metadata,hash 树 添加到 system 镜像末尾
build/tools/releasetools/build_image.py BuildVerifiedImage 生成最终的可以用于 dm-verity 校验的镜像
Append2Simg 添加 verity_image 和 datameta.img 到 system.img 末尾
其他
FIXED_SALT 可以修改为自己的 salt
如何生成自己的oem key
dm-verity 相关的密钥
build/target/product/security/ build/target/product/verity.mk verity.pk8 – 私钥,用于签名 boot.img 和 system.img
verity.x509.pem – 包含公钥的证书
verity_key – 公钥,dm verity 中用于验签 system 分区
- HSM(Hardware Security Module)
- OpenSSL tool
- 生成新得密钥对 >openssl version OpenSSL 1.0.2d 9 Jul 2015 >development/tools/make_key mykey ‘/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com’
- 为DM-Verity 功能生成 verity key
- 使用下面的命令来生成 verity key 的工具 generate_verity_key: source build/envsetup.sh choosecombo make generate_verity_key (mmm system/extras/verity/)
- 将 *.x509.pem 转换成 verity key generate_verity_key 的代码位于:system/extra/verity/generate_verity_key.c generate_verity_key 的用法:generate_verity_key | -convert 实例: out/host/linux-x86/bin/generate_verity_key -convert mykey.x509.pem verity_key
- 拷贝并重命名
- 拷贝pk8,mykey.x509.pem,verity_key.pub 至 build/target/product/security/ 目录,将其重命名: verity.pk8, verity.x509.pem,verity_key ,并替换默认的开发 key。
- 生成 keystore
- 执行下面的两个命令,生成img openssl rsa -in mykey.pk8 -inform DER -pubout -outform DER -out mypub.der java -Xmx512M -jar out/host/linux-x86/framework/KeystoreSigner.jar mykey.pk8 mykey.x509.pem keystore.img mypub.der
- 通过下面的脚本将img生成oem_keystore.h文件,shell 输入: function generate_oem_keystore_h() { echo \#ifndef __OEM_KEYSTORE_H echo \#define __OEM_KEYSTORE_H xxd -i $1 | sed -e ‘s/unsigned char .* = {/const unsigned char OEM_KEYSTORE[] = {/g’ -e ‘s/unsigned int .* =.*;//g’ echo \#endif }
- generate_oem_keystore_h keystore.img > oem_keystore.h
- 将该h文件拷贝到:bootable/bootloader/lk/platform/msm_shared/include
相关文件:
system/core/fs_mgr/fs_mgr_verity.c
用户空间对 dm-verity 进行初始化,验签 hash_table 的签名,传入 hash_table 等参数
相关文件:
KERNEL_SRC/driver/md/dm-verity.c 相关代码
内核空间根据用户空间传入的参数,进行初始化
当有 IO 操作时,对相应 data block 进行验签。
data block 读取时,不仅当前 data block 需要 hash 校验,上一层的 hash block 也需要进行校验,直到root hash。
每一个 data block 如果已经验签过,再次读取时就不用进行层层的校验,只校验当前 data block 的 hash 是否正确即可。
- kernel 配置文件使能 CONFIG_DM_VERITY
kernel/arch/arm/configs/msm8909-1gb_defconfig
- Android 相关 mk 文件配置
device/qcom/msm8909/msm8909.mk PRODUCT_SUPPORTS_VERITY := true PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/bootdevice/by-name/system
- 更新 fstab,system 分区添加 verify 标志
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait,verify
- 编译 userdebug 或者 user 版本 boot.img 以及 system.img,烧录
Mount | grep system /dev/block/dm-0 /system ext4 ro, seclable, relatime, data=ordered 0 0
adb pull /fstab.qcom c:\temp\fstab, 检查 verify 标志: /dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait, verify
adb remount Use “adb disable-verity” to disable verity. If you do not, remount may succeed, however, you will still not be able to write to these volumes. remount of system failed: Read-only file system remount failed
(1).先烧录启用 verity 功能的 boot.img 和 system.img
(2). 然后烧录未启用 verity 功能的 boot.img,重启后 push 一个 apk 到 /system/app/ 目录
(3). 重新烧录启用了 verity 功能的 boot.img,重启后,机器无法开机,kernel log 可以看到 data block xxx is corrupted
- 开机检验出错,无法开机
- 开机检验没问题,读取文件出错,返回 Error IO
软件集成和 OTA 升级,必须使用 block 方式生成 OTA 升级包,需要注意适配。
Verified Boot
https://source.android.com/security/verifiedboot/index.html
Verifying Boot
https://source.android.com/security/verifiedboot/verified-boot.html
Implementing dm-verity
https://source.android.com/security/verifiedboot/dm-verity.html
Dmverity
https://gitlab.com/cryptsetup/cryptsetup/wikis/DMVerity
dm-table format
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/device-mapper/verity.txt
Device mapper
https://www.ibm.com/developerworks/cn/linux/l-devmapper/
网友遇到的问题1:
问题现象:
fs_mgr: Failed to get verity metadata ‘/dev/block/bootdevice/by-name/system’ (Invalid argument)
解决方案:
博主 Kevin 也遇到这个问题了。这个问题在6.0以上的 Android 上可能会遇到。主要是实际 partition size 和 BoardConfig.mk 中的 system image size 的两个数值的大小不匹配,而且数值必须被4K整除。
解决方法:
1. 修改相关的分区表大小,以及 BoardConfig.mk 中 BOARD_SYSTEMIMAGE_PARTITION_SIZE 为相同的数值,并且必须被4K整除。
2. 如果还是有问题,那还要修改 system/extras/libfec/fec_open.cpp
int fec_open(struct fec_handle **handle, const char *path, int mode, int flags,
f->data_size = f->size; /* until ecc and/or verity are loaded */, change this to
f->data_size = 1610612736; /* until ecc and/or verity are loaded */
1610612736修改为具体项目的数值,如此这般,问题应该就可以解决了。
牛人。。。。。。。。。。。。。
过奖了,工作需要,深入研究了一下。
请教博主个问题,我现在按照你的启用步骤打开了verity,但是系统现在启动到init进程的起不来了。 可以帮忙分析下什么原因不? log 如下:
fs_mgr: Failed to get verity metadata ‘/dev/block/platform/fe330000.sdhci/by-name/system’ (No such file or directory)
[ 2.401986] fs_mgr: Could not set up verified partition, skipping!
[ 2.402054] fs_mgr: check_fs(): mount(/dev/block/platform/fe330000.sdhci/by-name/cache,/cache,ext4)=-1: No such file or directory
[ 2.402076] fs_mgr: Not running /system/bin/e2fsck on /dev/block/platform/fe330000.sdhci/by-name/cache (executable not in system image)
[ 2.414721] EXT4-fs (mmcblk1p9): recovery complete
[ 2.415166] EXT4-fs (mmcblk1p9): mounted filesystem with ordered data mode. Opts: noauto_da_alloc,discard
[ 2.415268] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/cache,target=/cache,type=ext4)=0
[ 2.415644] EXT4-fs (mmcblk1p11): Ignoring removed nomblk_io_submit option
[ 2.416543] EXT4-fs (mmcblk1p11): warning: maximal mount count reached, running e2fsck is recommended
[ 2.417003] EXT4-fs (mmcblk1p11): recovery complete
[ 2.417012] EXT4-fs (mmcblk1p11): mounted filesystem with ordered data mode. Opts: errors=remount-ro,nomblk_io_submit
[ 2.417046] fs_mgr: check_fs(): mount(/dev/block/platform/fe330000.sdhci/by-name/metadata,/metadata,ext4)=0: Success
[ 2.433835] fs_mgr: check_fs(): unmount(/metadata) succeeded
[ 2.433880] fs_mgr: Not running /system/bin/e2fsck on /dev/block/platform/fe330000.sdhci/by-name/metadata (executable not in system image)
[ 2.435099] EXT4-fs (mmcblk1p11): warning: maximal mount count reached, running e2fsck is recommended
[ 2.435633] EXT4-fs (mmcblk1p11): mounted filesystem with ordered data mode. Opts: noauto_da_alloc,discard
[ 2.435678] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/metadata,target=/metadata,type=ext4)=0
[ 2.435739] fs_mgr: Running /system/bin/fsck.f2fs -a /dev/block/platform/fe330000.sdhci/by-name/userdata
[ 2.436788] fsck.f2fs: executing /system/bin/fsck.f2fs failed: No such file or directory
[ 2.436788]
[ 2.436817] fsck.f2fs: fsck.f2fs terminated by exit(255)
[ 2.436817]
[ 2.514452] fs_mgr: __mount(source=/dev/block/platform/fe330000.sdhci/by-name/userdata,target=/data,type=f2fs)=0
[ 2.516245] init: (Loading properties from /system/build.prop took 0.00s.)
[ 2.516293] init: (Loading properties from /vendor/build.prop took 0.00s.)
[ 2.516322] init: (Loading properties from /factory/factory.prop took 0.00s.)
[ 2.516529] init: /recovery not specified in fstab
[ 2.516595] init: Starting service ‘healthd’…
[ 2.517042] init: cannot find ‘/system/vendor/bin/pvrsrvctl’ (No such file or directory), disabling ‘pvrsrvctl’
[ 2.517074] init: cannot find ‘/system/bin/displayd’ (No such file or directory), disabling ‘displayd’
[ 2.517111] init: cannot find ‘/system/vendor/bin/elogs.sh’ (No such file or directory), disabling ‘earlylogs’
[ 2.517147] init: cannot find ‘/system/bin/app_process64’ (No such file or directory), disabling ‘zygote’
[ 2.517177] init: cannot find ‘/system/bin/app_process32’ (No such file or directory), disabling ‘zygote_secondary’
[ 2.519257] binder: 191:191 transaction failed 29189, size 0-0
[ 2.520217] init: insmod: open(“/system/lib/modules/ump.ko”) failed: No such file or directoryinit: insmod: open(“/system/lib/modules/mali.y
[ 2.521509] zram0: detected capacity change from 0 to 533413888
[ 2.523097] Unable to find swap-space signature
[ 2.523215] fs_mgr: swapon failed for /dev/block/zram0
[ 2.523673] init: do_start: Service debuggerd not found
[ 2.523693] init: do_start: Service debuggerd64 not found
[ 2.523710] init: do_start: Service vold not found
[ 2.524740] init: Not bootcharting.
[ 2.560286] init: cannot find ‘/system/bin/tzdatacheck’ (No such file or directory), disabling ‘exec 1 (/system/bin/tzdatacheck)’
[ 2.563376] audit: type=1400 audit(1358534124.043:3): avc: denied { setattr } for pid=1 comm=”init” name=”cifsmanager” dev=”mmcblk1p13″ 1
[ 2.579632] F2FS-fs (mmcblk1p13): acl options not supported
[ 2.579665] F2FS-fs (mmcblk1p13): Unrecognized mount option “errors=panic” or missing value
[ 2.584165] init: write_file: Unable to open ‘/proc/sys/kernel/core_pattern’: No such file or directory
[ 2.584325] init: (Loading properties from /data/local.prop took 0.00s.)
[ 2.584560] init: do_start: Service logd not found
[ 2.584578] init: do_start: Service logd-reinit not found
[ 2.585272] init: write_file: Unable to open ‘/proc/sys/vm/min_free_order_shift’: No such file or directory
[ 2.586928] init: SELinux: Could not stat /sys/devices/system/cpu/cpufreq/interactive: No such file or directory.
[ 2.588051] init: write_file: Unable to open ‘/proc/sys/vm/lazy_vfree_tlb_flush_all_threshold’: No such file or directory
[ 2.591053] init: cannot find ‘/system/xbin/vm’ (No such file or directory), disabling ‘vm_daemon’
[ 2.592748] init: property ‘ro.serialno’ doesn’t exist while expanding ‘${ro.serialno}’
[ 2.592787] init: write: cannot expand ‘${ro.serialno}’
[ 2.592813] init: property ‘ro.product.manufacturer’ doesn’t exist while expanding ‘${ro.product.manufacturer}’
[ 2.592829] init: write: cannot expand ‘${ro.product.manufacturer}’
[ 2.592856] init: property ‘ro.product.model’ doesn’t exist while expanding ‘${ro.product.model}’
[ 2.592873] init: write: cannot expand ‘${ro.product.model}’
[ 2.593471] file system registered
[ 2.594175] using random self ethernet address
[ 2.594188] using random host ethernet address
[ 2.594251] init: write_file: Unable to open ‘/config/usb_gadget/g1/functions/rndis.gs4/wceis’: Permission denied
[ 2.601401] init: cannot find ‘/system/bin/sh’ (No such file or directory), disabling ‘console’
[ 2.601524] init: Starting service ‘adbd’…
[ 2.602305] init: cannot find ‘/system/bin/update_verifier’ (No such file or directory), disabling ‘exec 2 (/system/bin/update_verifier)’
[ 2.602342] init: cannot find ‘/system/bin/rild’ (No such file or directory), disabling ‘ril-daemon’
[ 2.602365] init: cannot find ‘/system/bin/install-recovery.sh’ (No such file or directory), disabling ‘flash_recovery’
[ 2.602395] init: cannot find ‘/system/bin/drmservice’ (No such file or directory), disabling ‘drmservice’
[ 2.602418] init: cannot find ‘/system/bin/bplus_helper’ (No such file or directory), disabling ‘bplus_helper’
[ 2.602439] init: cannot find ‘/system/bin/busybox’ (No such file or directory), disabling ‘up_eth0’
[ 2.602475] init: cannot find ‘/system/bin/akmd’ (No such file or directory), disabling ‘akmd’
[ 2.602515] init: cannot find ‘/system/bin/mediaserver’ (No such file or directory), disabling ‘media’
[ 2.602542] init: cannot find ‘/system/vendor/bin/start_log_srv.sh’ (No such file or directory), disabling ‘ap_log_srv’
[ 2.602564] init: cannot find ‘/system/vendor/bin/crashlogd’ (No such file or directory), disabling ‘crashlogd’
[ 2.602591] init: cannot find ‘/vendor/bin/log-watch’ (No such file or directory), disabling ‘log-watch’
[ 2.603471] init: write_file: Unable to open ‘/sys/class/android_usb/android0/enable’: Permission denied
[ 2.603531] init: write_file: Unable to open ‘/sys/class/android_usb/android0/idVendor’: Permission denied
[ 2.603587] init: write_file: Unable to open ‘/sys/class/android_usb/android0/idProduct’: Permission denied
[ 2.603638] init: write_file: Unable to open ‘/sys/class/android_usb/android0/functions’: Permission denied
[ 2.603683] init: write_file: Unable to open ‘/sys/class/android_usb/android0/enable’: Permission denied
[ 3.325195] mmc2: error -110 whilst initialising SDIO card
[ 3.338777] mmc_host mmc2: Bus speed (slot 0) = 300000Hz (slot req 300000Hz, actual 300000HZ div = 0)
[ 3.519480] binder: 191:191 transaction failed 29189, size 0-0
[ 4.071991] phy phy-ff770000.syscon:usb2-phy@e450.6: charger = USB_SDP_CHARGER
[ 4.174370] fusb302 4-0022: PD disabled
[ 4.176531] cdn-dp fec00000.dp: [drm:cdn_dp_pd_event_work] Not connected. Disabling cdn
[ 4.190663] rockchip-dwc3 usb@fe800000: USB peripheral connected
我用的是rockchip的rk3399平台android7.1
抱歉,回复晚了。不知道你的问题解决没有。
fs_mgr: Failed to get verity metadata ‘/dev/block/platform/fe330000.sdhci/by-name/system’ (No such file or directory)
这段log看起来是 dm-verify 生成的签名信息没有生成到 system.img 中,所以启动的时候没有找到 verify metadata。verity metadata 是镜像生成的时候附件到 system.img 尾部的,所以你需要确认一下你是否已经成功的启用了 dm-verity,并且重新 clean, 生成了正确的 system.img 。
terry,你好
这个问题解决了?可否分享下方法
分析的很全面,不错!
谢谢~~~
博主我也遇到这个问题了。这个问题在6.0以上的 Android 上可能会遇到。主要是实际 partition size 和 BoardConfig.mk 中的 system image size 的两个数值的大小不匹配,而且数值必须被4K整除。
解决方法:
1. 修改相关的分区表大小,以及 BoardConfig.mk 中 BOARD_SYSTEMIMAGE_PARTITION_SIZE 为相同的数值,并且必须被4K整除。
2. 如果还是有问题,那还要修改 system/extras/libfec/fec_open.cpp
int fec_open(struct fec_handle **handle, const char *path, int mode, int flags,
f->data_size = f->size; /* until ecc and/or verity are loaded */, change this to
f->data_size = 1610612736; /* until ecc and/or verity are loaded */
1610612736修改为具体项目的数值,如此这般,问题应该就可以解决了。如此这般,问题应该就可以解决了。
博主,您好:
android7.1的是不是有些变化了?发现要生成keystone.img无法生成,这个是什么原因?
Error: Unable to access jarfile out/host/linux-x86/framework/KeystoreSigner.jar
7.1 我试过了,没有遇到你说的问题。
看看是不是其他问题引起的?
刚刚有个问题不知道怎么解决。
然后去百度搜索。。。
然后搜索到 Kevin 我自己的博客上有解决方案。。。
还是我的原创文章。。。
@_@
博主,您好:
android7.1遇到一个问题,望能帮忙解惑:ioctl(fd, DM_TABLE_LOAD, io)中io参数个数的限制在10个或11个,这个是为什么,我看5.1和8.0都没有这个限制。
在dm-verity.c中函数static int verity_ctr(struct dm_target *ti, unsigned argc, char **argv)
{ …
if (argc 11) {
ti->error = “Invalid argument count: 10-11 arguments required”;
r = -EINVAL;
goto bad;
}
}
实际IO的参数是20个:
fs_mgr: loading verity table: ‘1 /dev/block/bootdevice/by-name/system /dev/block/bootdevice/by-name/system 4096 4096 258044 258044 sha256 4b401d9f595c05c45e910692a4f0695445e951f1d27031d3acdf2ebfea156f0a aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7 9 use_fec_from_device /dev/block/bootdevice/by-name/system fec_start 260085 fec_blocks 260085 fec_roots 2 ignore_zero_blocks’
这个细节没有留意,具体的项目需求决定了研究的方向以及深入程度。最近工作较忙,稍后有空研究一下,或者你有答案了,欢迎再来给大家解答。