Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 88a4d549 authored by Paolo Valente's avatar Paolo Valente Committed by Razziell
Browse files

block/bfq: Port BFQ-v8r10 to Android 3.18 kernel



block: cgroups, kconfig, build bits for BFQ-v7r11-4.4.0

Update Kconfig.iosched and do the related Makefile changes to include
kernel configuration options for BFQ. Also increase the number of
policies supported by the blkio controller so that BFQ can add its
own.

Signed-off-by: default avatarPaolo Valente <paolo.valente@unimore.it>
Signed-off-by: default avatarArianna Avanzini <avanzini@google.com>

block: introduce the BFQ-v7r11 I/O sched for 4.4.0

The general structure is borrowed from CFQ, as much of the code for
handling I/O contexts. Over time, several useful features have been
ported from CFQ as well (details in the changelog in README.BFQ). A
(bfq_)queue is associated to each task doing I/O on a device, and each
time a scheduling decision has to be made a queue is selected and served
until it expires.

    - Slices are given in the service domain: tasks are assigned
      budgets, measured in number of sectors. Once got the disk, a task
      must however consume its assigned budget within a configurable
      maximum time (by default, the maximum possible value of the
      budgets is automatically computed to comply with this timeout).
      This allows the desired latency vs "throughput boosting" tradeoff
      to be set.

    - Budgets are scheduled according to a variant of WF2Q+, implemented
      using an augmented rb-tree to take eligibility into account while
      preserving an O(log N) overall complexity.

    - A low-latency tunable is provided; if enabled, both interactive
      and soft real-time applications are guaranteed a very low latency.

    - Latency guarantees are preserved also in the presence of NCQ.

    - Also with flash-based devices, a high throughput is achieved
      while still preserving latency guarantees.

    - BFQ features Early Queue Merge (EQM), a sort of fusion of the
      cooperating-queue-merging and the preemption mechanisms present
      in CFQ. EQM is in fact a unified mechanism that tries to get a
      sequential read pattern, and hence a high throughput, with any
      set of processes performing interleaved I/O over a contiguous
      sequence of sectors.

    - BFQ supports full hierarchical scheduling, exporting a cgroups
      interface.  Since each node has a full scheduler, each group can
      be assigned its own weight.

    - If the cgroups interface is not used, only I/O priorities can be
      assigned to processes, with ioprio values mapped to weights
      with the relation weight = IOPRIO_BE_NR - ioprio.

    - ioprio classes are served in strict priority order, i.e., lower
      priority queues are not served as long as there are higher
      priority queues.  Among queues in the same class the bandwidth is
      distributed in proportion to the weight of each queue. A very
      thin extra bandwidth is however guaranteed to the Idle class, to
      prevent it from starving.

Signed-off-by: default avatarPaolo Valente <paolo.valente@unimore.it>
Signed-off-by: default avatarArianna Avanzini <avanzini@google.com>

block, bfq: add Early Queue Merge (EQM) to BFQ-v7r11 for 4.4.0

A set of processes may happen  to  perform interleaved reads, i.e.,requests
whose union would give rise to a  sequential read  pattern.  There are two
typical  cases: in the first  case,   processes  read  fixed-size chunks of
data at a fixed distance from each other, while in the second case processes
may read variable-size chunks at  variable distances. The latter case occurs
for  example with  QEMU, which  splits the  I/O generated  by the  guest into
multiple chunks,  and lets these chunks  be served by a  pool of cooperating
processes,  iteratively  assigning  the  next  chunk of  I/O  to  the first
available  process. CFQ  uses actual  queue merging  for the  first type of
rocesses, whereas it  uses preemption to get a sequential  read pattern out
of the read requests  performed by the second type of  processes. In the end
it uses  two different  mechanisms to  achieve the  same goal: boosting the
throughput with interleaved I/O.

This patch introduces  Early Queue Merge (EQM), a unified mechanism to get a
sequential  read pattern  with both  types of  processes. The  main idea is
checking newly arrived requests against the next request of the active queue
both in case of actual request insert and in case of request merge. By doing
so, both the types of processes can be handled by just merging their queues.
EQM is  then simpler and  more compact than the  pair of mechanisms used in
CFQ.

Finally, EQM  also preserves the  typical low-latency properties of BFQ, by
properly restoring the weight-raising state of a queue when it gets back to
a non-merged state.

Signed-off-by: default avatarMauro Andreolini <mauro.andreolini@unimore.it>
Signed-off-by: default avatarArianna Avanzini <avanzini@google.com>
Signed-off-by: default avatarPaolo Valente <paolo.valente@unimore.it>
Signed-off-by: default avatarLinus Walleij <linus.walleij@linaro.org>

Remove useless includes from bfq.h

Reverse include order between blk.h and bfq.h

Align blk-cgroup code with CFQ, and fix related small bugs

BUGFIX: expire bfqq if needed in bfq_bfqq_move

If bfqq happens to be in service but empty when bfq_bfqq_move is invoked,
then bfqq is also still marked as busy and bfq_del_bfqq_busy is invoked.
This triggers the failure of the check that bfq_del_bfqq_busy is not
invoked on the in-service queue. In particular, this flow of operations
is imprecise also because it is better to expire and reschedule bfqq
if in service, so as to make sure that it continues to be served only
if it makes sense accordi to its new group memebrship.

This commit fixes these problems by just expring bfqq, as a preliminary
step, if bfqq is in service.

Improve logging functions

Remove apparently useless check on bfq_class_idle from dispatch_requests

Serve also async queues based only on budget consumption

In addition to simplifying the code, this allows a more
precise control on the bandwidth provided to async queues.

Add group logging and more BUG_ONs for groups

Turn next-in-service check into a WARNING, and improve comments on next_in_service

Improve use of async_charge_factor

Introduce preemption-based, idleless service guarantees

If several processes are competing for the device at the
same time, but all processes and groups have the same weight,
then the mechanism introduced by this commit enables BFQ to
guarantee the expected throughput distribution without ever
idling the device. Throughput is then much higher in this
common scenario.

This commit also refactors the long function bfq_add_request,
and improves and extends most comments.

Improve handling of bursts of queue activations

This commits improves the handling of bursts of queue
activations in the following aspects.
- It reduces false positives by introducing the following
  constraints:
	. all queues in a burst must belong to the same group
	. only queue creations, and not reactivations,
	  contribute to a burst
- It improves burst detection in the presence of queue merging,
  by moving detection from bfq_add_request, which is not invoked
  in case of an early merge, to bfq_set_request, which is always
  invoked on the arrival of a request
- It fixes the re-marking of the flag in_large_burst after a split

Improve EQM

This commit improves EQM as follows.
- It lets weight-raised queues preserve weight-raising
- It reduces false positives by allowing only non-weight-raised or very
  recently weight-raised queues to be merged
- It checks whether a queue has just been split by using a timestamp,
  instead of a flag whose value might remain unchanged for a long time
  if no request arrived or was served for the queue

Squash me

Port CFQ commits 3dde36dd and e9ce335d

Port part of:
. commit 3dde36dd "rework of seeky detection"
	- remove constanly-seeky flag too
	- do not port the part related to nonrot: it does not
	  seem so convincing, because it does not take
	  sequential accesses into account at all
. commit e9ce335d "fix a kbuild regression"
	- basically introduce BFQQ_CLOSE_THR for EQM

Improve peak-rate estimation and rate-based autotuning

This commit improves peak-rate estimation and autotuning of
the parameters related to the device rate.
- It makes the low-pass filter for peak-rate estimation more
  conservative
- It improves the threshold for the switch between device-speed
  classes
- It introduces a simpler rule for detecting a queue as slow
- It replaces the function bfq_bfqq_charge_full_budget with the much
  more precise function bfq_bfqq_charge_time

Improve weight-raising mechanism

This commit:
- reduces the budget for weight-raised queues and makes it constant:
  this reduces latencies for weight-raised queues
- postpones the beginning of the weight-raising period of soft real-time
  queues, to when these queues do start to be served: this helps soft
  real-time queues preserve weight-raising despite the short duration
  of their weight-raising period
- makes the evauation of the idle period of a queue more precise
- increases the weight coefficient for weight-raised queues

BUGFIX: fix service-tree selection for non-leaf entities

Add extra logging

BUGFIX: remove incorrect group-related BUG_ON

Improve time handling to reduce wraparound issues

Change default group weight

Add strict-guarantees tunable

If this tunable is set, then device-idling is forced whenever needed to
provide accurate service guarantees. CAVEAT: idling unconditionally may
even increase latencies, in case of processes that did stop doing I/O.

BUGFIX: fix wrong Kconfig.iosched dependency for BFQ_GROUP_IOSCHED

Replace max with more efficient max_t

Replace time_is_XXX_jiffies when appropriate

Improve some comments

Fix computation of queue weight as a function of ioprio

Let weight range from 80 to 10 as a function of ioprio, in steps of 10.

Improve consistency with new IO controller

Announce as v8-rc2

Fix conditional-compilation errors related to cgroups

Port commit 4ebc1c61d6185604c97fd0b0355ab668052044ab

Port commit 95e5d6f62693f27d9011ec307eb32c6126314ea3

Port commit bce6133b09013f70d41a678d262a12147ed43889

Port commit 563180a44b7d7978f44e9776eedfbbc550c2398d

Use just queue_lock in bfq_check_ioprio_change

Port commit d93a11f1cd890d4ea72f7cef75fac56801b099b3

Port commit 2da8de0bb799bf2bdfa893e5a1e294eb6bafba62

Port commits 322731ed0dd and d4aad7ff04d

Port commit 30d7b944

BUGFIX: Extend rcu locking in bfq_bic_update_cgroup

Drop unnecessary synchronize_rcu

BFQ v8-rc3

BUGFIX: fix typo in constant definition

Tell whether cgroups support is enabled at boot

Suggested-by: default avatarLee Tibbert <lee.tibbert@gmail.com>

BFQ v8-rc4

BUGFIX: removed ref-incrememnt typo from a log message

Improve code

BFQ v8-rc5

BUGFIX: remove a harmless but meaningless comparison

Removed stale fields and comments

Add missing comment

Remove stale field

Further code cleanup

Move inline and improve data-structure comments, remove stale fields

BFQ v8-rc6

BFQ v8

Change suspect BUG_ON and add more BUG_ONs

Changed weight type

Add weight-changed log message

Privilege soft rt apps against interactive

BFQ v8r1

Remove variables not used if tracing is disabled

Reported-by: default avatarLee Tibbert <lee.tibbert@gmail.com>

Port commit ae11889636: turn blkg_lookup_create into blkg_lookup

BFQ v8r2

Fix switch from soft rt to interactive

BFQ v8r3

Update copyrigths to acknowledge Linaro's support

Signed-off-by: default avatar <paolo.valente@linaro.org>

Port to v4.8, 3rd part: switch slice_timer and fifo_expire to ns

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Port to v4.8, 4th part: switch slice_idle and thinktime to ns

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix handling of wait_request state

The semantics of hrtimers makes the following assumption false
after invoking hrtimer_try_to_cancel: the timer results as non
active. Unfortunately this assumption was used in the previous
version of the code. This commits lets code comply with the
new semantics.

Fix and extend logs, make other micro-improvements

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Improve the peak-rate estimator

This commit is a complete rewrite of the peak-rate estimation algorithm.  It is
both an improvement and a simplification: in particular it replaces the
previous, less effective, stable and clear algorithm for estimating the peak
rate.  The previous algorihtm approximated the service rate using the
individual dispatch rates observed during the service slots of queues. As such,
it took into account not only just individual queue workloads, but also rather
short time intervals.  The new algorithm considers the global workload served
by the device, and computes the peak rate over much larger time intervals. This
makes the new algorihtm extremely more effective with queueing devices and, in
general, with devices with a fluctuating bandwidth, either physical or virtual.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Force one-request-at-a-time service scheme with strict_guarantees

Force device to serve one request at a time if strict_guarantees is true.
Forcing this service scheme is currently the ONLY way to guarantee that the
request service order enforced by the scheduler is respected by a queueing
device. Otherwise the device is free even to make some unlucky request wait
for as long as the device wishes.

Of course, serving one request at at time may cause loss of throughput.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix use of slice_idle and use smaller type

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Guarantee that wr is not lost for a spurious temporary merge

If a queue Q1 is merged with an already-shared queue Q2, then Q1
may easily be freed soon after this event. If this happens and
the merge lasts little, then Q1 may be created again at a time
instant at which it had still to enjoy weight raising. This
commit checks that, and restores weight raising for the queue
if it is the case.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix recovery of lost service for soft rt

BFQ tries to let soft real-time queues recover the service they lose when,
although weight-raised and backlogged, they are not set in service soon. This
recovery is important for a soft real-time queue, because its weight-raising
period is rather short, and thus a service hole can easily cause the queue the
have already lost weight raising when it eventually restarts to be served (and
therefore before the queue can prove that it deserves its short weight-raising
period to be moved forward).

Unfortunately, there were some mistakes in the formula for computing the extent
of the service holes, which caused occasional overcompensations of holes. This
commit fixes the formula.

Signed-off-by: default avatarLuca Miccio <lucmiccio@gmail.com>

Let soft-rt wr start also while interactive wr is still in place

The quality of the service provided to a soft real-time application (e.g.,
audio/video playback) is maximum if the application starts to enjoy soft-rt
weight raising as it has finished its start-up. In this respect, to guarantee
the shortest-possible start-up time to applications, BFQ lets just-created
applications enjoy (a milder) weight-raising for a while. During this first
weight-raising period, which we can refer to as 'interactive', the application
could not switch to soft real-time weight raising, even if it met the
requirements ifor the switch, i.e., even if the actual start-up had finished,
and the application had started doing its soft real-time job.

This commits allows soft real-time weight raising to start for a queue, and
thus for the associated application, even while the queue is still enjoying
interactive weight raising.

This anticipated switch to soft real-time weight raising must be managed with
a little care.  In case of a spurious, temporary switch, a queue may unjustly
lose any form of weight raising when it then loses soft real-time weight
raising. If the queue is associated with an application which, e.g., is
actually still starting up, then BFQ will not help the application start
quickly any longer.  To avoid this problem, the interactive weight-raising
state of a queue is restored if the queue loses its soft real-time weight
raising when its interactive weight-raising period is not yet over.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Switch to ns for all rate timestamps, fixing also 32-bit issues

Signed-off-by: default avatarLuca Miccio <lucmiccio@gmail.com>
Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

block: bfq: fix Oops at system shutdown

Commit fbca70d33072 ("Port commit ae11889636: turn blkg_lookup_create into
blkg_lookup") introduced the possibility of bfq_find_set_group returning a
NULL pointer. When this happened in __bfq_bic_change_cgroup,
it caused an Oops a few lines later. This Oops was reported as reliably
happening on one machine on the sync_bfqq path at system shutdown.

This patch restores the prior behavior of falling back to using the
bfqd->root_group if bfqg is NULL. The behavior is restored, but at
a different place.

Note Well: This patch assumes that the prior behavior was correct.
     	   That assumption may not be warranted.

Signed-off-by: default avatarLee Tibbert <LeeTibbert@users.noreply.github.com>

BFQ-v8r4

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Documentation/block: add bfq-iosched.txt

Documentation of BFQ benefits, inner workings, interface and tunables.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGFIX: Replace max wrongly used for modulo numbers

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Improve bfq-iosched.txt

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Improve help message in Kconfig.iosched

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Remove stray disk word in first line

SIgned-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

bfq-iosched.txt: add description of preemption

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Add parentheses to complex macros

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix type in bfq-iosched.txt

Sigend-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r5

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Improve documentation

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Squash me

Add and improve pointers in initial comments

Fix typo in comments on bfq_lookup_next_entity

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix false-positive warning for uninitialized var

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Improve and fix some bugs in hierarchical code

This commit provides the following contributions.

BUGFIX Removed the check that, when the new queue to set in service
must be selected, the cached next_in_service entities coincide with
the entities chosen by __bfq_lookup_next_entity. This check,
issuing a warning on failure, was wrong, because the cached and the
newly chosen entity could differ in case of a CLASS_IDLE timeout.

EFFICIENCY IMPROVEMENT (this improvement is related to the above
BUGFIX) The cached next_in_service entities are now really used to
select the next queue to serve when the in-service queue
expires. Before this change, the cached values were used only for
extra (and in general wrong) consistency checks. This caused
additional overhead instead of reducing it.

EFFICIENCY IMPROVEMENT The next entity to serve, for each level of the
hierarchy, is now updated on every event that may change it, i.e., on
every activation or deactivation of any entity. This finer granularity
is not strictly needed for corectness, because it is only on queue
expirations that BFQ needs to know what are the next entities to
serve. Yet this change makes it possible to implement optimizations in
which it is necessary to know the next queue to serve before the
in-service queue expires.

SERVICE-ACCURACY IMPROVEMENT The per-device CLASS_IDLE service timeout
has been turned into a much more accurate per-group timeout.

CODE-QUALITY IMPROVEMENT The non-trivial parts touched by the above
improvements have been partially rewritten, and enriched of comments,
so as to improve their transparency and understandability.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ v8r6-rc1

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Port and improve CFQ commit 41647e7a

BFQ currently used the same logic for detecting seeky queues for
for rotational disks and SSDs. This logic is appropriate for the
former, as it takes into account only inter-request distance, and
this is the dominant latency factor on a rotational device. Yet
things change with flash-based devices, where servign a large
requests still yields a high throughput, even the request is far
from the previous request served. This commits extends seeky
detection to take into accoutn also this fact with flash-based
devices. In particular, this commit is an improved port of the
original commit 41647e7a for CFQ.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Provide more details in switching-off-wr message

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Remove useless parameter from bfq_del_bfqq_busy

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Further improve code and comments for hierarchical code

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Optimize update of next_in_service entity

If the update of the next_in_service candidate entity is triggered by
the activation of an entity, then it is not necessary to perform full
lookups in the active trees to update next_in_service. In fact, it is
enough to check whether the just-activated entity has a higher
priority of next_in_service, or, even if it has the same priority as
next_in_service, is eligible and has a lower virtual finish time than
next_in_service. If this compound condition holds, then the new entity
becomes the new next_in_service. Otherwise no change is needed. This
commit implements this optimization.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix bug causing occasional loss of weight raising

When a bfq_queue, say bfqq, is split after a merging with another
bfq_queue, BFQ checks whether it has to restore for bfqq the
weight-raising state that bfqq had before being merged.  In
particular, the weight-raising is restored only if, according to the
weight-raising duration decided for bfqq when it started to be
weight-raised (before being merged), bfqq would not have already
finished its weight-raising period.

Yet, by mistake, such a duration is not saved when bfqq is merged. So,
if bfqq is freed and reallocated when it is split, then this duration
is wrongly set to zero on the split. As a consequence, the
weight-raising state of bfqq is wrongly not restored, which causes BFQ
to fail in guaranteeing a low latency to bfqq.

This commit fixes this bug by saving weight-raising duration when bfqq
is merged, and correctly restoring it when bfqq is split.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix wrong reset of in-service entities

In-service entities were reset with an indirect logic, which
happened to be even buggy for some cases. This commit fixes
this bug in two important steps. First, by replacing this
indirect logic with a direct logic, in which all involved
entities are immediately reset, with a bubble-up loop, when
the in-service queue is reset. Second, by restructuring the
code related to this change, so as to become not only correct
with respect to this change, but also cleaner and hopefully
clearer.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Add code to redirect trace log to console

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ v8r6-rc2

Fix bug in optimized update of next_in_service entity

There was a case where bfq_update_next_in_service did not update
next_in_service, even if it might need to be changed: in case of
requeueing or repositioning of the entity that happened to be
pointed exactly by next_in_service. This could result in violation
of service guarantees, because, after a change of timestamps for
such an entity, it might be the case that next_in_service had to
point to a different entity. This commit fixes this bug.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Stop bubble-up of next_in_service update if possible

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ v8r6-rc3

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r6

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGFIX: make BFQ compile also without hierarchical support

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r7

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Remove wrong compilation warning

BUGFIX: Removed a wrong compilation warning, due to the compiler
not taking into account short circuit in a condition.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Add a ton of forgotten static qualifiers

BUGIFX: Added several forgotten static qualifiers in function
definitions (completely harmless issue).

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGFIX: Put async queues on exit also without cgroups

BUGFIX: The putting of async queues on scheduler exit was missing in case
cgroups support was not active. This fix adds the missing operation.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Fix check of the percentage of sequential dispatches

BUGFIX: In the peak-rate estimator, there was a serious error in the
check that the percentage of sequential I/O-request dispatches was high
enough to trigger an update of the peak-rate estimate. This commit fixes
that check.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Better tune weight-raising for slow flash-based devices

IMPROVEMENT Luca Miccio has run a few responsiveness tests on recent
Android systems with average-speed storage devices. These tests have
shown that the following BFQ parameter was too low for these systems:
reference duration for slow storage devices of weight raising for
interactive applications. This commit raises that duration to a
value that is yelding optimal results in our tests.

Signed-off-by: default avatarLuca Miccio <lucmiccio@gmail.com>
Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r8-rc1

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

Avoid a second dispatch in case of budget exhaustion

IMPROVEMENT This commit anticipates the complete check of budget exhaustion,
for the in-service bfq_queue, to when the next bfq_queue to serve is selected
(during a dispatch operation). This enables a new bfq_queue to be immediately
selected for service in case the in-service bfq_queue has actually exhausted
its budget. As a consequence, a second dispatch invocation is not needed
any more, to have a new request dispatched.
To implement this improvement, this commit implements a further improvement
too: the field next_rq of a bfq_queue now always contains the actual next
request to dispatch (or NULL if the bfq_queue is empty).

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r8-rc2

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

block/bfq-cgroup: fix bfq_bic_update_cgroup() API

bfq_bic_update_cgroup() should return nothing even if
CONFIG_BFQ_GROUP_IOSCHED is disabled.

Signed-off-by: default avatarOleksandr Natalenko <oleksandr@natalenko.name>

BFQ-v8r8

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGIFX: remove use of bfq queues after free

bfq queues occasionally happened to be used after being freed, because
they were accessed after some invocations of bfq_put_queue that could
cause them to be freed. This commit refactors code, when needed, to
avoid any occurrence of such a use-after-free of a bfq queue. This
commit also adds comments to make references to bfq queues easier to
follow.

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r9

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGFIX: Handle failure of weight-counter allocation

Reported-by: default avatarBart Van Assche <Bart.VanAssche@sandisk.com>
Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BUGFIX: Remove problematic check on max service duration

Reported-by: default avatarBart Van Assche <Bart.VanAssche@sandisk.com>
Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r10-rc1

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

BFQ-v8r10

Signed-off-by: default avatarPaolo Valente <paolo.valente@linaro.org>

block: bfq: Adjustments for 3.18 kernel

Change-Id: I4fdad52b2a2d54e5fffaa390328aad1b3ac03d99
Signed-off-by: default avatarAlbert I <krascgq@outlook.co.id>
parent 60b7f90e
Loading
Loading
Loading
Loading
+2 −0
Original line number Diff line number Diff line
00-INDEX
	- This file
bfq-iosched.txt
	- BFQ IO scheduler and its tunables
biodoc.txt
	- Notes on the Generic Block Layer Rewrite in Linux 2.5
capability.txt
+530 −0

File added.

Preview size limit exceeded, changes collapsed.

+11 −13
Original line number Diff line number Diff line
@@ -86,22 +86,20 @@ config IOSCHED_BFQ
	tristate "BFQ I/O scheduler"
	default n
	---help---
	  The BFQ I/O scheduler tries to distribute bandwidth among
	  all processes according to their weights.
	  It aims at distributing the bandwidth as desired, independently of
	  the disk parameters and with any workload. It also tries to
	  guarantee low latency to interactive and soft real-time
	  applications. If compiled built-in (saying Y here), BFQ can
	  be configured to support hierarchical scheduling.

config CGROUP_BFQIO
	The BFQ I/O scheduler distributes bandwidth among all
	processes according to their weights, regardless of the
	device parameters and with any workload. It also guarantees
	a low latency to interactive and soft real-time applications.
	Details in Documentation/block/bfq-iosched.txt

config BFQ_GROUP_IOSCHED
	bool "BFQ hierarchical scheduling support"
	depends on CGROUPS && IOSCHED_BFQ=y
	depends on IOSCHED_BFQ && BLK_CGROUP
	default n
	---help---
	  Enable hierarchical scheduling in BFQ, using the cgroups
	  filesystem interface.  The name of the subsystem will be
	  bfqio.

	Enable hierarchical scheduling in BFQ, using the blkio
	(cgroups-v1) or io (cgroups-v2) controller.

choice
	prompt "Default I/O scheduler"
+859 −610

File changed.

Preview size limit exceeded, changes collapsed.

+3 −3
Original line number Diff line number Diff line
@@ -14,7 +14,7 @@
 * icq_to_bic - convert iocontext queue structure to bfq_io_cq.
 * @icq: the iocontext queue.
 */
static inline struct bfq_io_cq *icq_to_bic(struct io_cq *icq)
static struct bfq_io_cq *icq_to_bic(struct io_cq *icq)
{
	/* bic->icq is the first member, %NULL will convert to %NULL */
	return container_of(icq, struct bfq_io_cq, icq);
@@ -27,7 +27,7 @@ static inline struct bfq_io_cq *icq_to_bic(struct io_cq *icq)
 *
 * Queue lock must be held.
 */
static inline struct bfq_io_cq *bfq_bic_lookup(struct bfq_data *bfqd,
static struct bfq_io_cq *bfq_bic_lookup(struct bfq_data *bfqd,
					struct io_context *ioc)
{
	if (ioc)
Loading