Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 44cee85a authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge branch 'misc' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild 

Pull misc kbuild updates from Michal Marek:

 - coccicheck script improvements by Luis Rodriguez and Deepa Dinamani

 - new coccinelle patches by Yann Droneaud and Vaishali Thakkar

 - debian packaging fixes by Wilfried Klaebe, Henning Schild and Marcin
   Mielniczuk

* 'misc' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
  Fix the Debian packaging script on systems with no codename
  builddeb: fix file permissions before packaging
  scripts/coccinelle: require coccinelle >= 1.0.4 on device_node_continue.cocci
  coccicheck: refer to Documentation/coccinelle.txt and wiki
  coccicheck: add support for requring a coccinelle version
  scripts: add Linux .cocciconfig for coccinelle
  coccicheck: replace --very-quiet with --quiet when debugging
  coccicheck: add support for DEBUG_FILE
  coccicheck: enable parmap support
  coccicheck: make SPFLAGS more useful
  coccicheck: move spatch binary check up
  builddeb: really include objtool binary in headers package
  coccinelle: catch krealloc() on devm_*() allocated memory
  coccinelle: recognize more devm_* memory allocation functions
  coccinelle: also catch kzfree() issues
  coccicheck: Allow for overriding spatch flags
  Coccinelle: noderef: Add new rules and correct the old rule
parents f716a85c 4c586062

.cocciconfig

0 → 100644
+3 −0
Original line number Original line Diff line number Diff line 
[spatch]

	options = --timeout 200

	options = --use-gitgrep

.gitignore

+1 −0
Original line number Original line Diff line number Diff line
@@ -67,6 +67,7 @@ Module.symvers 
#

#

!.gitignore

!.gitignore

!.mailmap

!.mailmap

!.cocciconfig





#

#

# Generated include files

# Generated include files

Documentation/coccinelle.txt

+147 −1
Original line number Original line Diff line number Diff line
@@ -38,6 +38,15 @@ as a regular user, and install it with 




        sudo make install

        sudo make install





 Supplemental documentation

~~~~~~~~~~~~~~~~~~~~~~~~~~~~



For supplemental documentation refer to the wiki:



https://bottest.wiki.kernel.org/coccicheck



The wiki documentation always refers to the linux-next version of the script.



 Using Coccinelle on the Linux kernel

 Using Coccinelle on the Linux kernel

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -94,11 +103,26 @@ To enable verbose messages set the V= variable, for example: 




   make coccicheck MODE=report V=1

   make coccicheck MODE=report V=1





 Coccinelle parallelization

~~~~~~~~~~~~~~~~~~~~~~~~~~~~



By default, coccicheck tries to run as parallel as possible. To change

By default, coccicheck tries to run as parallel as possible. To change

the parallelism, set the J= variable. For example, to run across 4 CPUs:

the parallelism, set the J= variable. For example, to run across 4 CPUs:





   make coccicheck MODE=report J=4

   make coccicheck MODE=report J=4





As of Coccinelle 1.0.2 Coccinelle uses Ocaml parmap for parallelization,

if support for this is detected you will benefit from parmap parallelization.



When parmap is enabled coccicheck will enable dynamic load balancing by using

'--chunksize 1' argument, this ensures we keep feeding threads with work

one by one, so that we avoid the situation where most work gets done by only

a few threads. With dynamic load balancing, if a thread finishes early we keep

feeding it more work.



When parmap is enabled, if an error occurs in Coccinelle, this error

value is propagated back, the return value of the 'make coccicheck'

captures this return value.





 Using Coccinelle with a single semantic patch

 Using Coccinelle with a single semantic patch

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -142,15 +166,118 @@ semantic patch as shown in the previous section. 
The "report" mode is the default. You can select another one with the

The "report" mode is the default. You can select another one with the

MODE variable explained above.

MODE variable explained above.





 Debugging Coccinelle SmPL patches

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Using coccicheck is best as it provides in the spatch command line

include options matching the options used when we compile the kernel.

You can learn what these options are by using V=1, you could then

manually run Coccinelle with debug options added.



Alternatively you can debug running Coccinelle against SmPL patches

by asking for stderr to be redirected to stderr, by default stderr

is redirected to /dev/null, if you'd like to capture stderr you

can specify the DEBUG_FILE="file.txt" option to coccicheck. For

instance:



    rm -f cocci.err

    make coccicheck COCCI=scripts/coccinelle/free/kfree.cocci MODE=report DEBUG_FILE=cocci.err

    cat cocci.err



You can use SPFLAGS to add debugging flags, for instance you may want to

add both --profile --show-trying to SPFLAGS when debugging. For instance

you may want to use:



    rm -f err.log

    export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci

    make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c



err.log will now have the profiling information, while stdout will

provide some progress information as Coccinelle moves forward with

work.



DEBUG_FILE support is only supported when using coccinelle >= 1.2.



 .cocciconfig support

~~~~~~~~~~~~~~~~~~~~~~



Coccinelle supports reading .cocciconfig for default Coccinelle options that

should be used every time spatch is spawned, the order of precedence for

variables for .cocciconfig is as follows:



  o Your current user's home directory is processed first

  o Your directory from which spatch is called is processed next

  o The directory provided with the --dir option is processed last, if used



Since coccicheck runs through make, it naturally runs from the kernel

proper dir, as such the second rule above would be implied for picking up a

.cocciconfig when using 'make coccicheck'.



'make coccicheck' also supports using M= targets.If you do not supply

any M= target, it is assumed you want to target the entire kernel.

The kernel coccicheck script has:



    if [ "$KBUILD_EXTMOD" = "" ] ; then

        OPTIONS="--dir $srctree $COCCIINCLUDE"

    else

        OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"

    fi



KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases

the spatch --dir argument is used, as such third rule applies when whether M=

is used or not, and when M= is used the target directory can have its own

.cocciconfig file. When M= is not passed as an argument to coccicheck the

target directory is the same as the directory from where spatch was called.



If not using the kernel's coccicheck target, keep the above precedence

order logic of .cocciconfig reading. If using the kernel's coccicheck target,

override any of the kernel's .coccicheck's settings using SPFLAGS.



We help Coccinelle when used against Linux with a set of sensible defaults

options for Linux with our own Linux .cocciconfig. This hints to coccinelle

git can be used for 'git grep' queries over coccigrep. A timeout of 200

seconds should suffice for now.



The options picked up by coccinelle when reading a .cocciconfig do not appear

as arguments to spatch processes running on your system, to confirm what

options will be used by Coccinelle run:



      spatch --print-options-only



You can override with your own preferred index option by using SPFLAGS. Take

note that when there are conflicting options Coccinelle takes precedence for

the last options passed. Using .cocciconfig is possible to use idutils, however

given the order of precedence followed by Coccinelle, since the kernel now

carries its own .cocciconfig, you will need to use SPFLAGS to use idutils if

desired. See below section "Additional flags" for more details on how to use

idutils.



 Additional flags

 Additional flags

~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~





Additional flags can be passed to spatch through the SPFLAGS

Additional flags can be passed to spatch through the SPFLAGS

variable.

variable. This works as Coccinelle respects the last flags

given to it when options are in conflict.





    make SPFLAGS=--use-glimpse coccicheck

    make SPFLAGS=--use-glimpse coccicheck



Coccinelle supports idutils as well but requires coccinelle >= 1.0.6.

When no ID file is specified coccinelle assumes your ID database file

is in the file .id-utils.index on the top level of the kernel, coccinelle

carries a script scripts/idutils_index.sh which creates the database with



    mkid -i C --output .id-utils.index



If you have another database filename you can also just symlink with this

name.



    make SPFLAGS=--use-idutils coccicheck

    make SPFLAGS=--use-idutils coccicheck





Alternatively you can specify the database filename explicitly, for

instance:



    make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck



See spatch --help to learn more about spatch options.

See spatch --help to learn more about spatch options.





Note that the '--use-glimpse' and '--use-idutils' options

Note that the '--use-glimpse' and '--use-idutils' options
@@ -159,6 +286,25 @@ thus active by default. However, by indexing the code with 
one of these tools, and according to the cocci file used,

one of these tools, and according to the cocci file used,

spatch could proceed the entire code base more quickly.

spatch could proceed the entire code base more quickly.





 SmPL patch specific options

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



SmPL patches can have their own requirements for options passed

to Coccinelle. SmPL patch specific options can be provided by

providing them at the top of the SmPL patch, for instance:



// Options: --no-includes --include-headers



 SmPL patch Coccinelle requirements

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



As Coccinelle features get added some more advanced SmPL patches

may require newer versions of Coccinelle. If an SmPL patch requires

at least a version of Coccinelle, this can be specified as follows,

as an example if requiring at least Coccinelle >= 1.0.5:



// Requires: 1.0.5



 Proposing new semantic patches

 Proposing new semantic patches

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

scripts/coccicheck

+86 −10
Original line number Original line Diff line number Diff line 
#!/bin/bash

#!/bin/bash



# Linux kernel coccicheck

#

# Read Documentation/coccinelle.txt

#

#

# This script requires at least spatch

# This script requires at least spatch

# version 1.0.0-rc11.

# version 1.0.0-rc11.

#





DIR="$(dirname $(readlink -f $0))/.."

SPATCH="`which ${SPATCH:=spatch}`"

SPATCH="`which ${SPATCH:=spatch}`"





trap kill_running SIGTERM SIGINT

if [ ! -x "$SPATCH" ]; then

declare -a SPATCH_PID

    echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'

    exit 1

fi



SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')

SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)



USE_JOBS="no"

$SPATCH --help | grep "\-\-jobs" > /dev/null && USE_JOBS="yes"





# The verbosity may be set by the environmental parameter V=

# The verbosity may be set by the environmental parameter V=

# as for example with 'make V=1 coccicheck'

# as for example with 'make V=1 coccicheck'
@@ -25,7 +35,28 @@ else 
	NPROC="$J"

	NPROC="$J"

fi

fi





FLAGS="$SPFLAGS --very-quiet"

FLAGS="--very-quiet"



# You can use SPFLAGS to append extra arguments to coccicheck or override any

# heuristics done in this file as Coccinelle accepts the last options when

# options conflict.

#

# A good example for use of SPFLAGS is if you want to debug your cocci script,

# you can for instance use the following:

#

# $ export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci

# $ make coccicheck MODE=report DEBUG_FILE="all.err" SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c

#

# "--show-trying" should show you what rule is being processed as it goes to

# stdout, you do not need a debug file for that. The profile output will be

# be sent to stdout, if you provide a DEBUG_FILE the profiling data can be

# inspected there.

#

# --profile will not output if --very-quiet is used, so avoid it.

echo $SPFLAGS | egrep -e "--profile|--show-trying" 2>&1 > /dev/null

if [ $? -eq 0 ]; then

	FLAGS="--quiet"

fi





# spatch only allows include directories with the syntax "-I include"

# spatch only allows include directories with the syntax "-I include"

# while gcc also allows "-Iinclude" and "-include include"

# while gcc also allows "-Iinclude" and "-include include"
@@ -51,9 +82,14 @@ if [ "$KBUILD_EXTMOD" != "" ] ; then 
    OPTIONS="--patch $srctree $OPTIONS"

    OPTIONS="--patch $srctree $OPTIONS"

fi

fi





if [ ! -x "$SPATCH" ]; then

# You can override by using SPFLAGS

    echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'

if [ "$USE_JOBS" = "no" ]; then

    exit 1

	trap kill_running SIGTERM SIGINT

	declare -a SPATCH_PID

elif [ "$NPROC" != "1" ]; then

	# Using 0 should work as well, refer to _SC_NPROCESSORS_ONLN use on

	# https://github.com/rdicosmo/parmap/blob/master/setcore_stubs.c

	OPTIONS="$OPTIONS --jobs $NPROC --chunksize 1"

fi

fi





if [ "$MODE" = "" ] ; then

if [ "$MODE" = "" ] ; then
@@ -72,7 +108,7 @@ if [ "$MODE" = "chain" ] ; then 
	echo 'All available modes will be tried (in that order): patch, report, context, org'

	echo 'All available modes will be tried (in that order): patch, report, context, org'

    fi

    fi

elif [ "$MODE" = "report" -o "$MODE" = "org" ] ; then

elif [ "$MODE" = "report" -o "$MODE" = "org" ] ; then

    FLAGS="$FLAGS --no-show-diff"

    FLAGS="--no-show-diff $FLAGS"

fi

fi





if [ "$ONLINE" = "0" ] ; then

if [ "$ONLINE" = "0" ] ; then
@@ -82,7 +118,26 @@ if [ "$ONLINE" = "0" ] ; then 
    echo ''

    echo ''

fi

fi





run_cmd() {

run_cmd_parmap() {

	if [ $VERBOSE -ne 0 ] ; then

		echo "Running ($NPROC in parallel): $@"

	fi

	if [ "$DEBUG_FILE" != "/dev/null" -a "$DEBUG_FILE" != "" ]; then

		if [ -f $DEBUG_FILE ]; then

			echo "Debug file $DEBUG_FILE exists, bailing"

			exit

		fi

	else

		DEBUG_FILE="/dev/null"

	fi

	$@ 2>$DEBUG_FILE

	if [[ $? -ne 0 ]]; then

		echo "coccicheck failed"

		exit $?

	fi

}



run_cmd_old() {

	local i

	local i

	if [ $VERBOSE -ne 0 ] ; then

	if [ $VERBOSE -ne 0 ] ; then

		echo "Running ($NPROC in parallel): $@"

		echo "Running ($NPROC in parallel): $@"
@@ -97,6 +152,14 @@ run_cmd() { 
	wait

	wait

}

}





run_cmd() {

	if [ "$USE_JOBS" = "yes" ]; then

		run_cmd_parmap $@

	else

		run_cmd_old $@

	fi

}



kill_running() {

kill_running() {

	for i in $(seq 0 $(( NPROC - 1 )) ); do

	for i in $(seq 0 $(( NPROC - 1 )) ); do

		if [ $VERBOSE -eq 2 ] ; then

		if [ $VERBOSE -eq 2 ] ; then
@@ -106,10 +169,23 @@ kill_running() { 
	done

	done

}

}





# You can override heuristics with SPFLAGS, these must always go last

OPTIONS="$OPTIONS $SPFLAGS"



coccinelle () {

coccinelle () {

    COCCI="$1"

    COCCI="$1"





    OPT=`grep "Option" $COCCI | cut -d':' -f2`

    OPT=`grep "Option" $COCCI | cut -d':' -f2`

    REQ=`grep "Requires" $COCCI | cut -d':' -f2 | sed "s| ||"`

    REQ_NUM=$(echo $REQ | ${DIR}/scripts/ld-version.sh)

    if [ "$REQ_NUM" != "0" ] ; then

	    if [ "$SPATCH_VERSION_NUM" -lt "$REQ_NUM" ] ; then

		    echo "Skipping coccinele SmPL patch: $COCCI"

		    echo "You have coccinelle:           $SPATCH_VERSION"

		    echo "This SmPL patch requires:      $REQ"

		    return

	    fi

    fi





#   The option '--parse-cocci' can be used to syntactically check the SmPL files.

#   The option '--parse-cocci' can be used to syntactically check the SmPL files.

#

#

scripts/coccinelle/free/devm_free.cocci

+26 −0
Original line number Original line Diff line number Diff line
@@ -29,7 +29,23 @@ expression x; 
@@

@@





(

(

 x = devm_kmalloc(...)

|

 x = devm_kvasprintf(...)

|

 x = devm_kasprintf(...)

|

 x = devm_kzalloc(...)

 x = devm_kzalloc(...)

|

 x = devm_kmalloc_array(...)

|

 x = devm_kcalloc(...)

|

 x = devm_kstrdup(...)

|

 x = devm_kmemdup(...)

|

 x = devm_get_free_pages(...)

|

|

 x = devm_request_irq(...)

 x = devm_request_irq(...)

|

|
@@ -48,6 +64,16 @@ position p; 
(

(

* kfree@p(x)

* kfree@p(x)

|

|

* kzfree@p(x)

|

* __krealloc@p(x, ...)

|

* krealloc@p(x, ...)

|

* free_pages@p(x, ...)

|

* free_page@p(x)

|

* free_irq@p(x)

* free_irq@p(x)

|

|

* iounmap@p(x)

* iounmap@p(x)