Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 37641374 authored by Akilesh Kailash's avatar Akilesh Kailash
Browse files

libsnapshot:VABC: Allow batch merge 



Kernel will batch the merge operations only when
block numbers of source and cow device are contiguous.

Daemon will read the COW file and post-process
the operations so that kernel can batch merge the potential
operations.

There are two key changes done in post-processing:

1: COW file contains all the copy operations at the
beginning of the file. We cannot allow batch
merging of COPY operations as a crash in between
the overlapping copies can result in a corrupted state.
Hence, allow copying individual blocks at a time.

2: Replace and Zero operations can be batch merged.
However, reading our existing COW format as-is
will not allow kernel to batch merge. Hence,
process the operations in such a way that kernel can batch
merge them.

Furthermore, it is observed that sync() after every
merge is a overhead. Hence, we will only sync()
for copy operations only. For replace and zero
operations, we will not explicitly sync. This is ok as
crash in between replace/zero merge operations can
redo those operations. However for copy, we have
to make sure that sync is completed before next copy
operation is initiated.

Merge time of a full OTA on bramble is around ~60
seconds as compared to ~10+ minutes prior to this
optimization.

Note that we still have copy operations which are not
batch merged. Hence, OTA with significant number of
copy operations can still have overhead on merge timings.

Bug: 174112589

Test: vts_libsnapshot, cow_snapuserd_test
Full OTA on bramble.

Signed-off-by: default avatarAkilesh Kailash <akailash@google.com>
Change-Id: I1dc286067a26ea399fa5d4e8e826e5622ce3fa58
parent 333639e9

fs_mgr/libsnapshot/cow_reader.cpp

+127 −18
Original line number Diff line number Diff line
@@ -189,27 +189,136 @@ bool CowReader::ParseOps(std::optional<uint64_t> label) { 
        LOG(INFO) << "No COW Footer, recovered data";

    }



    if (header_.num_merge_ops > 0) {

        uint64_t merge_ops = header_.num_merge_ops;

        uint64_t metadata_ops = 0;

        uint64_t current_op_num = 0;



        CHECK(ops_buffer->size() >= merge_ops);

        while (merge_ops) {

            auto& current_op = ops_buffer->data()[current_op_num];

            if (current_op.type == kCowLabelOp || current_op.type == kCowFooterOp) {

                metadata_ops += 1;

            } else {

                merge_ops -= 1;

    ops_ = ops_buffer;

    return true;

}

            current_op_num += 1;



void CowReader::InitializeMerge() {

    uint64_t num_copy_ops = 0;



    // Remove all the metadata operations

    ops_->erase(std::remove_if(ops_.get()->begin(), ops_.get()->end(),

                               [](CowOperation& op) {

                                   return (op.type == kCowFooterOp || op.type == kCowLabelOp);

                               }),

                ops_.get()->end());



    // We will re-arrange the vector in such a way that

    // kernel can batch merge. Ex:

    //

    // Existing COW format; All the copy operations

    // are at the beginning.

    // =======================================

    // Copy-op-1    - cow_op->new_block = 1

    // Copy-op-2    - cow_op->new_block = 2

    // Copy-op-3    - cow_op->new_block = 3

    // Replace-op-4 - cow_op->new_block = 6

    // Replace-op-5 - cow_op->new_block = 4

    // Replace-op-6 - cow_op->new_block = 8

    // Replace-op-7 - cow_op->new_block = 9

    // Zero-op-8    - cow_op->new_block = 7

    // Zero-op-9    - cow_op->new_block = 5

    // =======================================

    //

    // First find the operation which isn't a copy-op

    // and then sort all the operations in descending order

    // with the key being cow_op->new_block (source block)

    //

    // The data-structure will look like:

    //

    // =======================================

    // Copy-op-1    - cow_op->new_block = 1

    // Copy-op-2    - cow_op->new_block = 2

    // Copy-op-3    - cow_op->new_block = 3

    // Replace-op-7 - cow_op->new_block = 9

    // Replace-op-6 - cow_op->new_block = 8

    // Zero-op-8    - cow_op->new_block = 7

    // Replace-op-4 - cow_op->new_block = 6

    // Zero-op-9    - cow_op->new_block = 5

    // Replace-op-5 - cow_op->new_block = 4

    // =======================================

    //

    // Daemon will read the above data-structure in reverse-order

    // when reading metadata. Thus, kernel will get the metadata

    // in the following order:

    //

    // ========================================

    // Replace-op-5 - cow_op->new_block = 4

    // Zero-op-9    - cow_op->new_block = 5

    // Replace-op-4 - cow_op->new_block = 6

    // Zero-op-8    - cow_op->new_block = 7

    // Replace-op-6 - cow_op->new_block = 8

    // Replace-op-7 - cow_op->new_block = 9

    // Copy-op-3    - cow_op->new_block = 3

    // Copy-op-2    - cow_op->new_block = 2

    // Copy-op-1    - cow_op->new_block = 1

    // ===========================================

    //

    // When merging begins, kernel will start from the last

    // metadata which was read: In the above format, Copy-op-1

    // will be the first merge operation.

    //

    // Now, batching of the merge operations happens only when

    // 1: origin block numbers in the base device are contiguous

    // (cow_op->new_block) and,

    // 2: cow block numbers which are assigned by daemon in ReadMetadata()

    // are contiguous. These are monotonically increasing numbers.

    //

    // When both (1) and (2) are true, kernel will batch merge the operations.

    // However, we do not want copy operations to be batch merged as

    // a crash or system reboot during an overlapping copy can drive the device

    // to a corrupted state. Hence, merging of copy operations should always be

    // done as a individual 4k block. In the above case, since the

    // cow_op->new_block numbers are contiguous, we will ensure that the

    // cow block numbers assigned in ReadMetadata() for these respective copy

    // operations are not contiguous forcing kernel to issue merge for each

    // copy operations without batch merging.

    //

    // For all the other operations viz. Replace and Zero op, the cow block

    // numbers assigned by daemon will be contiguous allowing kernel to batch

    // merge.

    //

    // The final format after assiging COW block numbers by the daemon will

    // look something like:

    //

    // =========================================================

    // Replace-op-5 - cow_op->new_block = 4  cow-block-num = 2

    // Zero-op-9    - cow_op->new_block = 5  cow-block-num = 3

    // Replace-op-4 - cow_op->new_block = 6  cow-block-num = 4

    // Zero-op-8    - cow_op->new_block = 7  cow-block-num = 5

    // Replace-op-6 - cow_op->new_block = 8  cow-block-num = 6

    // Replace-op-7 - cow_op->new_block = 9  cow-block-num = 7

    // Copy-op-3    - cow_op->new_block = 3  cow-block-num = 9

    // Copy-op-2    - cow_op->new_block = 2  cow-block-num = 11

    // Copy-op-1    - cow_op->new_block = 1  cow-block-num = 13

    // ==========================================================

    //

    // Merge sequence will look like:

    //

    // Merge-1 - Copy-op-1

    // Merge-2 - Copy-op-2

    // Merge-3 - Copy-op-3

    // Merge-4 - Batch-merge {Replace-op-7, Replace-op-6, Zero-op-8,

    //                        Replace-op-4, Zero-op-9, Replace-op-5 }

    //==============================================================



    for (uint64_t i = 0; i < ops_->size(); i++) {

        auto& current_op = ops_->data()[i];

        if (current_op.type != kCowCopyOp) {

            break;

        }

        ops_buffer->erase(ops_buffer.get()->begin(),

                          ops_buffer.get()->begin() + header_.num_merge_ops + metadata_ops);

        num_copy_ops += 1;

    }



    ops_ = ops_buffer;

    return true;

    std::sort(ops_.get()->begin() + num_copy_ops, ops_.get()->end(),

              [](CowOperation& op1, CowOperation& op2) -> bool {

                  return op1.new_block > op2.new_block;

              });



    if (header_.num_merge_ops > 0) {

        CHECK(ops_->size() >= header_.num_merge_ops);

        ops_->erase(ops_.get()->begin(), ops_.get()->begin() + header_.num_merge_ops);

    }

}



bool CowReader::GetHeader(CowHeader* header) {

fs_mgr/libsnapshot/cow_writer.cpp

+6 −2
Original line number Diff line number Diff line
@@ -421,7 +421,7 @@ bool CowWriter::Sync() { 
    return true;

}



bool CowWriter::CommitMerge(int merged_ops) {

bool CowWriter::CommitMerge(int merged_ops, bool sync) {

    CHECK(merge_in_progress_);

    header_.num_merge_ops += merged_ops;
@@ -436,8 +436,12 @@ bool CowWriter::CommitMerge(int merged_ops) { 
        return false;

    }



    // Sync only for merging of copy operations.

    if (sync) {

        return Sync();

    }

    return true;

}



bool CowWriter::Truncate(off_t length) {

    if (is_dev_null_ || is_block_device_) {

fs_mgr/libsnapshot/include/libsnapshot/cow_reader.h

+2 −0
Original line number Diff line number Diff line
@@ -140,6 +140,8 @@ class CowReader : public ICowReader { 


    void UpdateMergeProgress(uint64_t merge_ops) { header_.num_merge_ops += merge_ops; }



    void InitializeMerge();



  private:

    bool ParseOps(std::optional<uint64_t> label);

fs_mgr/libsnapshot/include/libsnapshot/cow_writer.h

+1 −1
Original line number Diff line number Diff line
@@ -98,7 +98,7 @@ class CowWriter : public ICowWriter { 
    bool InitializeAppend(android::base::borrowed_fd fd, uint64_t label);



    void InitializeMerge(android::base::borrowed_fd fd, CowHeader* header);

    bool CommitMerge(int merged_ops);

    bool CommitMerge(int merged_ops, bool sync);



    bool Finalize() override;

fs_mgr/libsnapshot/include/libsnapshot/snapuserd.h

+2 −3
Original line number Diff line number Diff line
@@ -81,7 +81,7 @@ class Snapuserd final { 
    bool ReadDiskExceptions(chunk_t chunk, size_t size);

    bool ReadData(chunk_t chunk, size_t size);

    bool IsChunkIdMetadata(chunk_t chunk);

    chunk_t GetNextAllocatableChunkId(chunk_t chunk);

    chunk_t GetNextAllocatableChunkId(chunk_t chunk_id);



    bool ProcessReplaceOp(const CowOperation* cow_op);

    bool ProcessCopyOp(const CowOperation* cow_op);
@@ -90,8 +90,7 @@ class Snapuserd final { 
    loff_t GetMergeStartOffset(void* merged_buffer, void* unmerged_buffer,

                               int* unmerged_exceptions);

    int GetNumberOfMergedOps(void* merged_buffer, void* unmerged_buffer, loff_t offset,

                             int unmerged_exceptions);

    bool AdvanceMergedOps(int merged_ops_cur_iter);

                             int unmerged_exceptions, bool* copy_op);

    bool ProcessMergeComplete(chunk_t chunk, void* buffer);

    sector_t ChunkToSector(chunk_t chunk) { return chunk << CHUNK_SHIFT; }

    chunk_t SectorToChunk(sector_t sector) { return sector >> CHUNK_SHIFT; }