Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 3ea911bc authored by Akilesh Kailash's avatar Akilesh Kailash
Browse files

snapuserd: Add I/O path support for variable block size 



The flow of I/O path is as follows:

1: When there is a I/O request for a given sector, we first
check the in-memory COW operation mapping for that sector.

2: If the mapping of sector to COW operation is found, then the
existing I/O path will work seamlessly. Even if the COW operation
encodes multiple blocks, we will discard the remaining data.

3: If the mapping of sector to COW operation is not found:
    a: Find the previous COW operation as the vector has sorted sectors.

    b: If the previous COW operation is a REPLACE op:
        i: Check if the current sector is encoded in the previous COW
	operations compressed block.

	ii: If the sector falls within the range of compressed blocks,
	retrieve the block offset.

	iii: De-compress the COW operation based on the compression
	factor.

	iv: memcpy the data based on the block offset.

	v: cache the COW operation pointer as subsequent I/O requests
	are sequential and can just be a memcpy at the correct offset.
    c: If the previous COW operation is not a REPLACE op or if the
       requested sector does not fall within the compression factor
       of the previous COW operation, then fallback and read the data
       from base device.

Snapshot-merge:

During merge of REPLACE ops, read the entire op in one shot, de-compress
multiple blocks and write all the blocks in one shot.

Performance:

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

Bug: 319309466

Test: snapuserd_test covers all the I/O path with various block sizes.
About 252 cases with all combinations and tunables.

[==========] 252 tests from 4 test suites ran. (702565 ms total)
[  PASSED  ] 252 tests.

On Pixel 6:

=======================================
COW Writer V3:

for i in full, incremental OTA
   for j in 4k, 16k, 32k, 64k, 128, 256k
      for k in lz4, zstd, gz
	 install OTA, reboot, verify merge
=======================================
COW Writer V2:

for i in full, incremental OTA
  for j in 4k
    for k in lz4, zstd, gz
      install OTA, reboot, verity merge

=====================================

Change-Id: I4c3b5c3efa0d09677568b4396cc53db0e74e7c99
Signed-off-by: default avatarAkilesh Kailash <akailash@google.com>
parent 59fa4867

fs_mgr/libsnapshot/include/libsnapshot/cow_format.h

+4 −0
Original line number Diff line number Diff line
@@ -359,5 +359,9 @@ std::optional<CowCompressionAlgorithm> CompressionAlgorithmFromString(std::strin 


// Return block size used for compression

size_t CowOpCompressionSize(const CowOperation* op, size_t block_size);



// Return the relative offset of the I/O block which the CowOperation

// multi-block compression

bool GetBlockOffset(const CowOperation* op, uint64_t io_block, size_t block_size, off_t* offset);

}  // namespace snapshot

}  // namespace android

fs_mgr/libsnapshot/include/libsnapshot/cow_reader.h

+3 −0
Original line number Diff line number Diff line
@@ -162,6 +162,9 @@ class CowReader final : public ICowReader { 
    // Creates a clone of the current CowReader without the file handlers

    std::unique_ptr<CowReader> CloneCowReader();



    // Get the max compression size

    uint32_t GetMaxCompressionSize();



    void UpdateMergeOpsCompleted(int num_merge_ops) { header_.num_merge_ops += num_merge_ops; }



  private:

fs_mgr/libsnapshot/libsnapshot_cow/cow_compress.cpp

+1 −1
Original line number Diff line number Diff line
@@ -223,7 +223,7 @@ bool CompressWorker::CompressBlocks(ICompressor* compressor, size_t block_size, 
            PLOG(ERROR) << "CompressBlocks: Compression failed";

            return false;

        }

        if (data.size() > std::numeric_limits<uint16_t>::max()) {

        if (data.size() > std::numeric_limits<uint32_t>::max()) {

            LOG(ERROR) << "Compressed block is too large: " << data.size();

            return false;

        }

fs_mgr/libsnapshot/libsnapshot_cow/cow_format.cpp

+26 −0
Original line number Diff line number Diff line
@@ -181,5 +181,31 @@ size_t CowOpCompressionSize(const CowOperation* op, size_t block_size) { 
    return (block_size << compression_bits);

}



bool GetBlockOffset(const CowOperation* op, uint64_t io_block, size_t block_size, off_t* offset) {

    const uint64_t new_block = op->new_block;



    if (op->type() != kCowReplaceOp || io_block < new_block) {

        LOG(VERBOSE) << "Invalid IO request for block: " << io_block

                     << " CowOperation: new_block: " << new_block;

        return false;

    }



    // Get the actual compression size

    const size_t compression_size = CowOpCompressionSize(op, block_size);

    // Find the number of blocks spanned

    const size_t num_blocks = compression_size / block_size;

    // Find the distance of the I/O block which this

    // CowOperation encompasses

    const size_t block_distance = io_block - new_block;

    // Check if this block is within this range;

    // if so, return the relative offset

    if (block_distance < num_blocks) {

        *offset = block_distance * block_size;

        return true;

    }



    return false;

}



}  // namespace snapshot

}  // namespace android

fs_mgr/libsnapshot/libsnapshot_cow/cow_reader.cpp

+15 −0
Original line number Diff line number Diff line
@@ -164,6 +164,21 @@ bool CowReader::Parse(android::base::borrowed_fd fd, std::optional<uint64_t> lab 
    return PrepMergeOps();

}



uint32_t CowReader::GetMaxCompressionSize() {

    switch (header_.prefix.major_version) {

        case 1:

        case 2:

            // Old versions supports only 4KB compression.

            return header_.block_size;

            ;

        case 3:

            return header_.max_compression_size;

        default:

            LOG(ERROR) << "Unknown version: " << header_.prefix.major_version;

            return 0;

    }

}



//

// This sets up the data needed for MergeOpIter. MergeOpIter presents

// data in the order we intend to merge in.