[PATCH 00/11] travis: try to reduce failures

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[PATCH 00/11] travis: try to reduce failures

Philippe Mathieu-Daudé
These patches try to improve our Travis CI usage (quite a few failures the last
days). This series include Peter's "Automate coverity scan uploads via Travis"
patches (https://www.mail-archive.com/qemu-devel@.../msg457443.html).

Patch 1 is a no-op as of today, but if eventually Shippable improve their
hardware the builds will profit of it directly.

Patch 2 allow to check how many time is spent building the codebase versus
running the tests.

Patch 4/5 speed up a bit Travis building many test objects in parallel (using
all available cores) previous to run the tests sequentially (as before) so the
tests output don't get mixed.

Patch 6 use 'travis_retry' feature to avoid aborting a job on network failure
retrying 3 times (default).

Then are Peter patches improving Coverity.

Regards,

Phil.

Peter Maydell (3):
  travis: install more library dependencies
  scripts/run-coverity-scan: Script to run Coverity Scan build
  travis: Add config to do a Coverity Scan upload

Philippe Mathieu-Daudé (8):
  travis: build using all available cpus
  travis: split building/testing to have finer elapsed time
  travis: use yes/no variable to enable/disable tests
  tests: add rule to compile many objects used by tests
  travis: build tests objects in parallel, then run tests sequentially
  travis: retry if llvm.org timeouts
  travis: install more library dependencies
  MAINTAINERS: self-appoint me as reviewer in build/test automation

 .travis.yml               |  82 ++++++++++++++++++----
 MAINTAINERS               |   1 +
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/Makefile.include    |   8 ++-
 4 files changed, 244 insertions(+), 17 deletions(-)
 create mode 100755 scripts/run-coverity-scan

--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 01/11] travis: build using all available cpus

Philippe Mathieu-Daudé
no improvement as of today, but if Travis release their limit on the opensource
plan or upgrade their hardware the builds will get some benefit.

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 27a2d9cfb3..d21a2a3602 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -45,6 +45,7 @@ notifications:
     on_failure: always
 env:
   global:
+    - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
     - TEST_CMD="make check"
   matrix:
     - CONFIG=""
@@ -64,7 +65,7 @@ before_install:
 before_script:
   - ./configure ${CONFIG}
 script:
-  - make -j3 && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 02/11] travis: split building/testing to have finer elapsed time

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
to check each step elapsed time on the travis output report.

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index d21a2a3602..b2c69fdd59 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -65,7 +65,8 @@ before_install:
 before_script:
   - ./configure ${CONFIG}
 script:
-  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS}
+  - test -z "${TEST_CMD}" || ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 03/11] travis: use yes/no variable to enable/disable tests

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
having the command to run in the variable make scripts harder to manage.

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---

using 0/1 or no/yes is pretty much the same.

 .travis.yml | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index b2c69fdd59..c9ac741afc 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -46,7 +46,7 @@ notifications:
 env:
   global:
     - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
-    - TEST_CMD="make check"
+    - RUN_TESTS="yes"
   matrix:
     - CONFIG=""
     - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
@@ -66,7 +66,7 @@ before_script:
   - ./configure ${CONFIG}
 script:
   - make -j${PARALLEL_JOBS}
-  - test -z "${TEST_CMD}" || ${TEST_CMD}
+  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
 matrix:
   include:
     # Test with CLang for compile portability
@@ -77,16 +77,16 @@ matrix:
       compiler: gcc
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=simple"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=ftrace"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=ust"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG=""
       os: osx
@@ -191,6 +191,6 @@ matrix:
       env:
         - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
         - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
-        - TEST_CMD=""
+        - RUN_TESTS="no"
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 04/11] tests: add rule to compile many objects used by tests

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
all those objects can get compiled simultaneously

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 tests/Makefile.include | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index f42f3dfa72..3773f9d8d2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -804,6 +804,7 @@ check-help:
  @echo " make check-qapi-schema    Run QAPI schema tests"
  @echo " make check-block          Run block tests"
  @echo " make check-report.html    Generates an HTML test report"
+ @echo " make check-compile        Compile objects used by tests"
  @echo " make check-clean          Clean the tests"
  @echo
  @echo "Please note that HTML reports do not regenerate if the unit tests"
@@ -887,7 +888,7 @@ check-tests/qapi-schema/doc-good.texi: tests/qapi-schema/doc-good.test.texi
 
 # Consolidated targets
 
-.PHONY: check-qapi-schema check-qtest check-unit check check-clean
+.PHONY: check-qapi-schema check-qtest check-unit check check-clean check-compile
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-unit: $(patsubst %,check-%, $(check-unit-y))
@@ -897,7 +898,10 @@ check-clean:
  $(MAKE) -C tests/tcg clean
  rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
  rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
-
+check-compile: $(check-qtest-generic-y) $(check-unit-y) $(test-util-obj-y) $(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) $(chardev-obj-y) $(QEMU_IOTESTS_HELPERS-y)
+ifeq ($(CONFIG_SOFTMMU),y)
+check-compile: $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)))
+endif
 clean: check-clean
 
 # Build the help program automatically
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
tests are run sequentially to avoid mixed results output.

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index c9ac741afc..dbbb11617d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -66,7 +66,7 @@ before_script:
   - ./configure ${CONFIG}
 script:
   - make -j${PARALLEL_JOBS}
-  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
+  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
 matrix:
   include:
     # Test with CLang for compile portability
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 06/11] travis: retry if llvm.org timeouts

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857

    $ sudo apt-get update -qq
    W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    The command "sudo apt-get update -qq" failed and exited with 100 during .
    Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index dbbb11617d..69a960b714 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -123,10 +123,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
@@ -142,10 +142,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 07/11] travis: install more library dependencies

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
From: Peter Maydell <[hidden email]>

Update the travis list of library packages to install so that
our build tests cover more of our code base.

Signed-off-by: Peter Maydell <[hidden email]>
Reviewed-by: Alex Bennée <[hidden email]>
Reviewed-by: Philippe Mathieu-Daudé <[hidden email]>
Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 69a960b714..4889c192c2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,28 +11,37 @@ addons:
       # Build dependencies
       - libaio-dev
       - libattr1-dev
+      - libbluetooth-dev
       - libbrlapi-dev
+      - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
+      - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
+      - librdmacm-dev
       - libsdl1.2-dev
       - libseccomp-dev
+      - libsnappy-dev
       - libspice-protocol-dev
       - libspice-server-dev
       - libssh2-1-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvde-dev
       - libvte-2.90-dev
+      - libxen-dev
+      - nettle-dev
       - sparse
       - uuid-dev
+      - xfslibs-dev
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
 # to prevent IRC notifications from forks. This was created using:
@@ -164,28 +173,37 @@ matrix:
             # Build dependencies
             - libaio-dev
             - libattr1-dev
+            - libbluetooth-dev
             - libbrlapi-dev
+            - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
             - libgtk-3-dev
             - libiscsi-dev
             - liblttng-ust-dev
+            - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
+            - librdmacm-dev
             - libsdl1.2-dev
             - libseccomp-dev
+            - libsnappy-dev
             - libspice-protocol-dev
             - libspice-server-dev
             - libssh2-1-dev
             - liburcu-dev
             - libusb-1.0-0-dev
+            - libvde-dev
             - libvte-2.90-dev
+            - libxen-dev
+            - nettle-dev
             - sparse
             - uuid-dev
+            - xfslibs-dev
       language: generic
       compiler: none
       env:
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 08/11] travis: install more library dependencies

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
so more codebase is built

Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 4889c192c2..4c0f7f444e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,10 +9,12 @@ addons:
   apt:
     packages:
       # Build dependencies
+      - glusterfs-common
       - libaio-dev
       - libattr1-dev
       - libbluetooth-dev
       - libbrlapi-dev
+      - libbz2-dev
       - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
@@ -22,6 +24,8 @@ addons:
       - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
+      - libncursesw5-dev
+      - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
@@ -171,10 +175,12 @@ matrix:
             - gcc-5
             - g++-5
             # Build dependencies
+            - glusterfs-common
             - libaio-dev
             - libattr1-dev
             - libbluetooth-dev
             - libbrlapi-dev
+            - libbz2-dev
             - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
@@ -184,6 +190,8 @@ matrix:
             - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
+            - libncursesw5-dev
+            - libnfs-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 09/11] scripts/run-coverity-scan: Script to run Coverity Scan build

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
From: Peter Maydell <[hidden email]>

Add a new script to automate the process of running the Coverity
Scan build tools and uploading the resulting tarball to the
website. This is primarily intended to be driven from Travis,
but it can be run locally (if you are a maintainer of the
QEMU project on the Coverity Scan website and have the secret
upload token).

Signed-off-by: Peter Maydell <[hidden email]>
Reviewed-by: Alex Bennée <[hidden email]>
Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 170 insertions(+)
 create mode 100755 scripts/run-coverity-scan

diff --git a/scripts/run-coverity-scan b/scripts/run-coverity-scan
new file mode 100755
index 0000000000..e6d5fc58d8
--- /dev/null
+++ b/scripts/run-coverity-scan
@@ -0,0 +1,170 @@
+#!/bin/sh -e
+
+# Upload a created tarball to Coverity Scan, as per
+# https://scan.coverity.com/projects/qemu/builds/new
+
+# This work is licensed under the terms of the GNU GPL version 2,
+# or (at your option) any later version.
+# See the COPYING file in the top-level directory.
+#
+# Copyright (c) 2017 Linaro Limited
+# Written by Peter Maydell
+
+# Note that this script will automatically download and
+# run the (closed-source) coverity build tools, so don't
+# use it if you don't trust them!
+
+# This script assumes that you're running it from a QEMU source
+# tree, and that tree is a fresh clean one, because we do an in-tree
+# build. (This is necessary so that the filenames that the Coverity
+# Scan server sees are relative paths that match up with the component
+# regular expressions it uses; an out-of-tree build won't work for this.)
+# The host machine should have as many of QEMU's dependencies
+# installed as possible, for maximum coverity coverage.
+
+# You need to pass the following environment variables to the script:
+#  COVERITY_TOKEN -- this is the secret 8 digit hex string which lets
+#                    you upload to Coverity Scan. If you're a maintainer
+#                    in Coverity then the web UI will tell you this.
+#  COVERITY_EMAIL -- the email address to use for uploads
+
+# and optionally
+#  COVERITY_DRYRUN -- set to not actually do the upload
+#  COVERITY_BUILD_CMD -- make command (defaults to 'make -j8')
+#  COVERITY_TOOL_BASE -- set to directory to put coverity tools
+#                        (defaults to /tmp/coverity-tools)
+
+# The primary purpose of this script is to be run as part of
+# a Travis build, but it is possible to run it manually locally.
+
+if [ -z "$COVERITY_TOKEN" ]; then
+    echo "COVERITY_TOKEN environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_EMAIL" ]; then
+    echo "COVERITY_EMAIL environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_BUILD_CMD" ]; then
+    echo "COVERITY_BUILD_CMD: using default 'make -j8'"
+    COVERITY_BUILD_CMD="make -j8"
+fi
+
+if [ -z "$COVERITY_TOOL_BASE" ]; then
+    echo "COVERITY_TOOL_BASE: using default /tmp/coverity-tools"
+    COVERITY_TOOL_BASE=/tmp/coverity-tools
+fi
+
+PROJTOKEN="$COVERITY_TOKEN"
+PROJNAME=QEMU
+TARBALL=cov-int.tar.xz
+SRCDIR="$(pwd)"
+
+echo "Checking this is a QEMU source tree..."
+if ! [ -e VERSION ]; then
+    echo "Not in a QEMU source tree?"
+    exit 1
+fi
+
+echo "Checking upload permissions..."
+
+if ! up_perm="$(wget https://scan.coverity.com/api/upload_permitted --post-data "token=$PROJTOKEN&project=$PROJNAME" -q -O -)"; then
+    echo "Coverity Scan API access denied: bad token?"
+    exit 1
+fi
+
+# Really up_perm is a JSON response with either
+# {upload_permitted:true} or {next_upload_permitted_at:<date>}
+# We do some hacky string parsing instead of properly parsing it.
+case "$up_perm" in
+    *upload_permitted*true*)
+        echo "Coverity Scan: upload permitted"
+        ;;
+    *next_upload_permitted_at*)
+        if [ -z "$COVERITY_DRYRUN" ]; then
+            echo "Coverity Scan: upload quota reached; stopping here"
+            # Exit success as this isn't a build error.
+            exit 0
+        else
+            echo "Coverity Scan: upload quota reached, continuing dry run"
+        fi
+        ;;
+    *)
+        echo "Coverity Scan upload check: unexpected result $up_perm"
+        exit 1
+        ;;
+esac
+
+mkdir -p "$COVERITY_TOOL_BASE"
+cd "$COVERITY_TOOL_BASE"
+
+echo "Checking for new version of coverity build tools..."
+wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME&md5=1" -O coverity_tool.md5.new
+
+if ! cmp -s coverity_tool.md5 coverity_tool.md5.new; then
+    # out of date md5 or no md5: download new build tool
+    # blow away the old build tool
+    echo "Downloading coverity build tools..."
+    rm -rf coverity_tool coverity_tool.tgz
+    wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME" -O coverity_tool.tgz
+    if ! (cat coverity_tool.md5.new; echo "  coverity_tool.tgz") | md5sum -c --status; then
+        echo "Downloaded tarball didn't match md5sum!"
+        exit 1
+    fi
+    # extract the new one, keeping it corralled in a 'coverity_tool' directory
+    echo "Unpacking coverity build tools..."
+    mkdir -p coverity_tool
+    cd coverity_tool
+    tar xf ../coverity_tool.tgz
+    cd ..
+    mv coverity_tool.md5.new coverity_tool.md5
+fi
+
+rm -f coverity_tool.md5.new
+
+TOOLBIN="$(echo $(pwd)/coverity_tool/cov-analysis-*/bin)"
+
+if ! test -x "$TOOLBIN/cov-build"; then
+    echo "Couldn't find cov-build in the coverity build-tool directory??"
+    exit 1
+fi
+
+export PATH="$TOOLBIN:$PATH"
+
+cd "$SRCDIR"
+
+echo "Doing make distclean..."
+make distclean
+
+echo "Configuring..."
+./configure --audio-drv-list=oss,alsa,sdl,pa --disable-werror
+
+echo "Making libqemustub.a..."
+make libqemustub.a
+
+echo "Running cov-build..."
+rm -rf cov-int
+mkdir cov-int
+cov-build --dir cov-int $COVERITY_BUILD_CMD
+
+echo "Creating results tarball..."
+tar cvf - cov-int | xz > "$TARBALL"
+
+echo "Uploading results tarball..."
+
+VERSION="$(git describe --always HEAD)"
+DESCRIPTION="$(git rev-parse HEAD)"
+
+if ! [ -z "$COVERITY_DRYRUN" ]; then
+    echo "Dry run only, not uploading $TARBALL"
+    exit 0
+fi
+
+curl --form token="$PROJTOKEN" --form email="$COVERITY_EMAIL" \
+     --form file=@"$TARBALL" --form version="$VERSION" \
+     --form description="$DESCRIPTION" \
+     https://scan.coverity.com/builds?project="$PROJNAME"
+
+echo "Done."
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 10/11] travis: Add config to do a Coverity Scan upload

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
From: Peter Maydell <[hidden email]>

Add config to travis to do a Coverity Scan build and upload, using
the new run-coverity-scan script.

There is an official integration between Travis and Coverity Scan:
 https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
which slurps values out of the .travis.yml and downloads a build
script from Coverity which does the bulk of the work:
 https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh

However we choose to roll our own since this seems less
confusing and also allows us to include debug features
(notably the ability to do a "dry run" test which doesn't
actually upload anything).

Signed-off-by: Peter Maydell <[hidden email]>
Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
---
 .travis.yml | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 4c0f7f444e..fc91e2fd28 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -220,3 +220,27 @@ matrix:
         - RUN_TESTS="no"
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
+    # Build and upload to Coverity Scan.
+    # We do not impose any rate limiting here, but instead rely on the
+    # limiting done by the coverity servers, which for a project of QEMU's
+    # size means one build a day. The run-coverity-scan script will exit
+    # early if the limiter does not permit a new upload, so the effect will
+    # be that the first build (only) in each 24 hour period will be scanned.
+    # If we needed to apply a limit at the Travis end, the simplest approach
+    # would be to run the scan only if the branch was 'coverity-scan', and
+    # use a cron job to push master to the 'coverity-scan' branch periodically.
+    # We run on the trusty Travis hosts so that there's a wider set of
+    # dependencies satisfied to improve coverage.
+    - dist: trusty
+      env:
+        - COVERITY=1
+        - COVERITY_BUILD_CMD="make -j3"
+        - COVERITY_EMAIL=[hidden email]
+        # This 'secure' setting sets COVERITY_TOKEN=<secret token>
+        # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
+        - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
+      before_script:
+        - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
+        - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping Coverity (wrong branch)"; exit 0; fi
+      script:
+        - ./scripts/run-coverity-scan
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

[PATCH 11/11] MAINTAINERS: self-appoint me as reviewer in build/test automation

Philippe Mathieu-Daudé
In reply to this post by Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
Reviewed-by: Alex Bennée <[hidden email]>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 120788d8fb..e62cbc439a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1858,6 +1858,7 @@ Build and test automation
 -------------------------
 M: Alex Bennée <[hidden email]>
 M: Fam Zheng <[hidden email]>
+R: Philippe Mathieu-Daudé <[hidden email]>
 L: [hidden email]
 S: Maintained
 F: .travis.yml
--
2.11.0


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 03/11] travis: use yes/no variable to enable/disable tests

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> having the command to run in the variable make scripts harder to
> manage.

I'm not sure this adds much over the TEST_CMD not set == don't run we
had before.

>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
> ---
>
> using 0/1 or no/yes is pretty much the same.
>
>  .travis.yml | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index b2c69fdd59..c9ac741afc 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -46,7 +46,7 @@ notifications:
>  env:
>    global:
>      - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
> -    - TEST_CMD="make check"
> +    - RUN_TESTS="yes"
>    matrix:
>      - CONFIG=""
>      - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
> @@ -66,7 +66,7 @@ before_script:
>    - ./configure ${CONFIG}
>  script:
>    - make -j${PARALLEL_JOBS}
> -  - test -z "${TEST_CMD}" || ${TEST_CMD}
> +  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -77,16 +77,16 @@ matrix:
>        compiler: gcc
>      # We manually include builds which we disable "make check" for
>      - env: CONFIG="--enable-debug --enable-tcg-interpreter"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=simple"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=ftrace"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=ust"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG=""
>        os: osx
> @@ -191,6 +191,6 @@ matrix:
>        env:
>          - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
>          - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
> -        - TEST_CMD=""
> +        - RUN_TESTS="no"
>        before_script:
>          - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 01/11] travis: build using all available cpus

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> no improvement as of today, but if Travis release their limit on the opensource
> plan or upgrade their hardware the builds will get some benefit.
>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>

Reviewed-by: Alex Bennée <[hidden email]>

> ---
>  .travis.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 27a2d9cfb3..d21a2a3602 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -45,6 +45,7 @@ notifications:
>      on_failure: always
>  env:
>    global:
> +    - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
>      - TEST_CMD="make check"
>    matrix:
>      - CONFIG=""
> @@ -64,7 +65,7 @@ before_install:
>  before_script:
>    - ./configure ${CONFIG}
>  script:
> -  - make -j3 && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 02/11] travis: split building/testing to have finer elapsed time

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> to check each step elapsed time on the travis output report.
>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>

Reviewed-by: Alex Bennée <[hidden email]>


> ---
>  .travis.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index d21a2a3602..b2c69fdd59 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -65,7 +65,8 @@ before_install:
>  before_script:
>    - ./configure ${CONFIG}
>  script:
> -  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS}
> +  - test -z "${TEST_CMD}" || ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 04/11] tests: add rule to compile many objects used by tests

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> all those objects can get compiled simultaneously
>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
> ---
>  tests/Makefile.include | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index f42f3dfa72..3773f9d8d2 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -804,6 +804,7 @@ check-help:
>   @echo " make check-qapi-schema    Run QAPI schema tests"
>   @echo " make check-block          Run block tests"
>   @echo " make check-report.html    Generates an HTML test report"
> + @echo " make check-compile        Compile objects used by tests"
>   @echo " make check-clean          Clean the tests"
>   @echo
>   @echo "Please note that HTML reports do not regenerate if the unit tests"
> @@ -887,7 +888,7 @@ check-tests/qapi-schema/doc-good.texi: tests/qapi-schema/doc-good.test.texi
>
>  # Consolidated targets
>
> -.PHONY: check-qapi-schema check-qtest check-unit check check-clean
> +.PHONY: check-qapi-schema check-qtest check-unit check check-clean check-compile
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  check-unit: $(patsubst %,check-%, $(check-unit-y))
> @@ -897,7 +898,10 @@ check-clean:
>   $(MAKE) -C tests/tcg clean
>   rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>   rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> -
> +check-compile: $(check-qtest-generic-y) $(check-unit-y) $(test-util-obj-y) $(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) $(chardev-obj-y) $(QEMU_IOTESTS_HELPERS-y)
> +ifeq ($(CONFIG_SOFTMMU),y)
> +check-compile: $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)))
> +endif
>  clean: check-clean
>
>  # Build the help program automatically

Something has gone wrong here:

  11:54 alex@zen taken:111, git:review/travis-speedup, [/home/alex/lsrc/qemu/qemu.git]> make check-compile
          CHK version_gen.h
    LINK    tests/ac97-test
  tests/ac97-test.o: In function `main':
  /home/alex/lsrc/qemu/qemu.git/tests/ac97-test.c:23: undefined reference to `qtest_add_func'
  tests/ac97-test.o: In function `qtest_start':
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:521: undefined reference to `qtest_init'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:521: undefined reference to `global_qtest'
  tests/ac97-test.o: In function `qtest_end':
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:532: undefined reference to `global_qtest'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:532: undefined reference to `qtest_quit'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:533: undefined reference to `global_qtest'
  collect2: error: ld returned 1 exit status
  /home/alex/lsrc/qemu/qemu.git/rules.mak:121: recipe for target 'tests/ac97-test' failed
  make: *** [tests/ac97-test] Error 1

Yet "make check" completes without issue.

Also I would expect the check: target to have check-compile as one of
its pre-requisites otherwise this target will bit rot.

--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> tests are run sequentially to avoid mixed results output.

I'm not so sure we are worried about mixed results output. I don't think
anyone can compare the entire log and they tend to scroll to the end
where the error will be.

>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>
> ---
>  .travis.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c9ac741afc..dbbb11617d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -66,7 +66,7 @@ before_script:
>    - ./configure ${CONFIG}
>  script:
>    - make -j${PARALLEL_JOBS}
> -  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
> +  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 06/11] travis: retry if llvm.org timeouts

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857
>
>     $ sudo apt-get update -qq
>     W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
>     E: Some index files failed to download. They have been ignored, or old ones used instead.
>     The command "sudo apt-get update -qq" failed and exited with 100 during .
>     Your build has been stopped.
>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>

This is good - I suspect there are other places we could use travis_retry.

Reviewed-by: Alex Bennée <[hidden email]>

> ---
>  .travis.yml | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index dbbb11617d..69a960b714 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -123,10 +123,10 @@ matrix:
>          - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
>          - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
>        before_install:
> -        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> -        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> -        - sudo apt-get update -qq
> -        - sudo apt-get install -qq -y clang-3.9
> +        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> +        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> +        - travis_retry sudo apt-get update -qq
> +        - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
> @@ -142,10 +142,10 @@ matrix:
>          - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
>          - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
>        before_install:
> -        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> -        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> -        - sudo apt-get update -qq
> -        - sudo apt-get install -qq -y clang-3.9
> +        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> +        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> +        - travis_retry sudo apt-get update -qq
> +        - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 08/11] travis: install more library dependencies

Alex Bennée-2
In reply to this post by Philippe Mathieu-Daudé

Philippe Mathieu-Daudé <[hidden email]> writes:

> so more codebase is built
>
> Signed-off-by: Philippe Mathieu-Daudé <[hidden email]>

Acked-by: Alex Bennée <[hidden email]>

> ---
>  .travis.yml | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 4889c192c2..4c0f7f444e 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -9,10 +9,12 @@ addons:
>    apt:
>      packages:
>        # Build dependencies
> +      - glusterfs-common
>        - libaio-dev
>        - libattr1-dev
>        - libbluetooth-dev
>        - libbrlapi-dev
> +      - libbz2-dev
>        - libcap-dev
>        - libcap-ng-dev
>        - libgnutls-dev
> @@ -22,6 +24,8 @@ addons:
>        - liblzo2-dev
>        - libnfs-dev
>        - libncurses5-dev
> +      - libncursesw5-dev
> +      - libnfs-dev
>        - libnss3-dev
>        - libpixman-1-dev
>        - libpng12-dev
> @@ -171,10 +175,12 @@ matrix:
>              - gcc-5
>              - g++-5
>              # Build dependencies
> +            - glusterfs-common
>              - libaio-dev
>              - libattr1-dev
>              - libbluetooth-dev
>              - libbrlapi-dev
> +            - libbz2-dev
>              - libcap-dev
>              - libcap-ng-dev
>              - libgnutls-dev
> @@ -184,6 +190,8 @@ matrix:
>              - liblzo2-dev
>              - libnfs-dev
>              - libncurses5-dev
> +            - libncursesw5-dev
> +            - libnfs-dev
>              - libnss3-dev
>              - libpixman-1-dev
>              - libpng12-dev


--
Alex Bennée

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially

Peter Maydell-5
In reply to this post by Alex Bennée-2
On 19 June 2017 at 11:58, Alex Bennée <[hidden email]> wrote:
>
> Philippe Mathieu-Daudé <[hidden email]> writes:
>
>> tests are run sequentially to avoid mixed results output.
>
> I'm not so sure we are worried about mixed results output. I don't think
> anyone can compare the entire log and they tend to scroll to the end
> where the error will be.

Indeed, and make's --output-sync flag should be enough to keep
individual test output distinct.

What is perhaps a problem is that I'm not sure our tests all support
being run in parallel with each other without tripping each other up by
using the same temporary file / TCP port / etc at once. To the
extent that this doesn't work it's a bug, but we should start
by doing a solid soak test of the test suite to make sure -j<bignum>
works reliably.

thanks
-- PMM

12