[PATCH for-5.0 v2 0/3] benchmark util

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH for-5.0 v2 0/3] benchmark util

Vladimir Sementsov-Ogievskiy-2
Hi all!

Here is simple benchmarking utility, to generate performance
comparison tables, like the following:

----------  -------------  -------------  -------------
            backup-1       backup-2       mirror
ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
----------  -------------  -------------  -------------

This is a v2, as v1 was inside
 "[RFC 00/24] backup performance: block_status + async"

I'll use this benchmark in other series, hope someone
will like it.

Vladimir Sementsov-Ogievskiy (3):
  python: add simplebench.py
  python: add qemu/bench_block_job.py
  python: add example usage of simplebench

 python/bench-example.py        |  80 +++++++++++++++++++++
 python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
 python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
 3 files changed, 323 insertions(+)
 create mode 100644 python/bench-example.py
 create mode 100755 python/qemu/bench_block_job.py
 create mode 100644 python/simplebench.py

--
2.18.0


Reply | Threaded
Open this post in threaded view
|

[PATCH for-5.0 v2 1/3] python: add simplebench.py

Vladimir Sementsov-Ogievskiy-2
Add simple benchmark table creator.

Signed-off-by: Vladimir Sementsov-Ogievskiy <[hidden email]>
---
 python/simplebench.py | 128 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 python/simplebench.py

diff --git a/python/simplebench.py b/python/simplebench.py
new file mode 100644
index 0000000000..59e7314ff6
--- /dev/null
+++ b/python/simplebench.py
@@ -0,0 +1,128 @@
+#!/usr/bin/env python
+#
+# Simple benchmarking framework
+#
+# Copyright (c) 2019 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
+    """Benchmark one test-case
+
+    test_func   -- benchmarking function with prototype
+                   test_func(env, case), which takes test_env and test_case
+                   arguments and returns {'seconds': int} (which is benchmark
+                   result) on success and {'error': str} on error. Returned
+                   dict may contain any other additional fields.
+    test_env    -- test environment - opaque first argument for test_func
+    test_case   -- test case - opaque second argument for test_func
+    count       -- how many times to call test_func, to calculate average
+    initial_run -- do initial run of test_func, which don't get into result
+
+    Returns dict with the following fields:
+        'runs':     list of test_func results
+        'average':  average seconds per run (exists only if at least one run
+                    succeeded)
+        'delta':    maximum delta between test_func result and the average
+                    (exists only if at least one run succeeded)
+        'n-failed': number of failed runs (exists only if at least one run
+                    failed)
+    """
+    if initial_run:
+        print('  #initial run:')
+        print('   ', test_func(test_env, test_case))
+
+    runs = []
+    for i in range(count):
+        print('  #run {}'.format(i+1))
+        res = test_func(test_env, test_case)
+        print('   ', res)
+        runs.append(res)
+
+    result = {'runs': runs}
+
+    successed = [r for r in runs if ('seconds' in r)]
+    if successed:
+        avg = sum(r['seconds'] for r in successed) / len(successed)
+        result['average'] = avg
+        result['delta'] = max(abs(r['seconds'] - avg) for r in successed)
+
+    if len(successed) < count:
+        result['n-failed'] = count - len(successed)
+
+    return result
+
+
+def ascii_one(result):
+    """Return ASCII representation of bench_one() returned dict."""
+    if 'average' in result:
+        s = '{:.2f} +- {:.2f}'.format(result['average'], result['delta'])
+        if 'n-failed' in result:
+            s += '\n({} failed)'.format(result['n-failed'])
+        return s
+    else:
+        return 'FAILED'
+
+
+def bench(test_func, test_envs, test_cases, *args, **vargs):
+    """Fill benchmark table
+
+    test_func -- benchmarking function, see bench_one for description
+    test_envs -- list of test environments, see bench_one
+    test_cases -- list of test cases, see bench_one
+    args, vargs -- additional arguments for bench_one
+
+    Returns dict with the following fields:
+        'envs':  test_envs
+        'cases': test_cases
+        'tab':   filled 2D array, where cell [i][j] is bench_one result for
+                 test_cases[i] for test_envs[j] (i.e., rows are test cases and
+                 columns are test environments)
+    """
+    tab = {}
+    results = {
+        'envs': test_envs,
+        'cases': test_cases,
+        'tab': tab
+    }
+    n = 1
+    n_tests = len(test_envs) * len(test_cases)
+    for env in test_envs:
+        for case in test_cases:
+            print('Testing {}/{}: {} :: {}'.format(n, n_tests,
+                                                   env['id'], case['id']))
+            if case['id'] not in tab:
+                tab[case['id']] = {}
+            tab[case['id']][env['id']] = bench_one(test_func, env, case,
+                                                   *args, **vargs)
+            n += 1
+
+    print('Done')
+    return results
+
+
+def ascii(results):
+    """Return ASCII representation of bench() returned dict."""
+    from tabulate import tabulate
+
+    tab = [[""] + [c['id'] for c in results['envs']]]
+    for case in results['cases']:
+        row = [case['id']]
+        for env in results['envs']:
+            row.append(ascii_one(results['tab'][case['id']][env['id']]))
+        tab.append(row)
+
+    return tabulate(tab)
--
2.18.0


Reply | Threaded
Open this post in threaded view
|

[PATCH for-5.0 v2 2/3] python: add qemu/bench_block_job.py

Vladimir Sementsov-Ogievskiy-2
In reply to this post by Vladimir Sementsov-Ogievskiy-2
Add block-job benchmarking helper functions.

Signed-off-by: Vladimir Sementsov-Ogievskiy <[hidden email]>
---
 python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)
 create mode 100755 python/qemu/bench_block_job.py

diff --git a/python/qemu/bench_block_job.py b/python/qemu/bench_block_job.py
new file mode 100755
index 0000000000..93f3956158
--- /dev/null
+++ b/python/qemu/bench_block_job.py
@@ -0,0 +1,115 @@
+#!/usr/bin/env python
+#
+# Benchmark block jobs
+#
+# Copyright (c) 2019 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+import socket
+import json
+from .machine import QEMUMachine
+from .qmp import QMPConnectError
+
+
+def bench_block_job(cmd, cmd_args, qemu_args):
+    """Benchmark block-job
+
+    cmd       -- qmp command to run block-job (like blockdev-backup)
+    cmd_args  -- dict of qmp command arguments
+    qemu_args -- list of Qemu command line arguments, including path to Qemu
+                 binary
+
+    Returns {'seconds': int} on success and {'error': str} on failure, dict may
+    contain addional 'vm-log' field. Return value is compatible with
+    simplebench lib.
+    """
+
+    vm = QEMUMachine(qemu_args[0], args=qemu_args[1:])
+
+    try:
+        vm.launch()
+    except OSError as e:
+        return {'error': 'popen failed: ' + str(e)}
+    except (QMPConnectError, socket.timeout):
+        return {'error': 'qemu failed: ' + str(vm.get_log())}
+
+    try:
+        res = vm.qmp(cmd, **cmd_args)
+        if res != {'return': {}}:
+            vm.shutdown()
+            return {'error': '"{}" command failed: {}'.format(cmd, str(res))}
+
+        e = vm.event_wait('JOB_STATUS_CHANGE')
+        assert e['data']['status'] == 'created'
+        start_ms = e['timestamp']['seconds'] * 1000000 + \
+            e['timestamp']['microseconds']
+
+        e = vm.events_wait((('BLOCK_JOB_READY', None),
+                            ('BLOCK_JOB_COMPLETED', None),
+                            ('BLOCK_JOB_FAILED', None)), timeout=True)
+        if e['event'] not in ('BLOCK_JOB_READY', 'BLOCK_JOB_COMPLETED'):
+            vm.shutdown()
+            return {'error': 'block-job failed: ' + str(e),
+                    'vm-log': vm.get_log()}
+        end_ms = e['timestamp']['seconds'] * 1000000 + \
+            e['timestamp']['microseconds']
+    finally:
+        vm.shutdown()
+
+    return {'seconds': (end_ms - start_ms) / 1000000.0}
+
+
+# Bench backup or mirror
+def bench_block_copy(qemu_binary, cmd, source, target):
+    """Helper to run bench_block_job() for mirror or backup"""
+    assert cmd in ('blockdev-backup', 'blockdev-mirror')
+
+    source['node-name'] = 'source'
+    target['node-name'] = 'target'
+
+    return bench_block_job(cmd,
+                           {'job-id': 'job0', 'device': 'source',
+                            'target': 'target', 'sync': 'full'},
+                           [qemu_binary,
+                            '-blockdev', json.dumps(source),
+                            '-blockdev', json.dumps(target)])
+
+
+def drv_file(filename):
+    return {'driver': 'file', 'filename': filename,
+            'cache': {'direct': True}, 'aio': 'native'}
+
+
+def drv_nbd(host, port):
+    return {'driver': 'nbd',
+            'server': {'type': 'inet', 'host': host, 'port': port}}
+
+
+if __name__ == '__main__':
+    import sys
+
+    if len(sys.argv) < 4:
+        print('USAGE: {} <qmp block-job command name> '
+              '<json string of arguments for the command> '
+              '<qemu binary path and arguments>'.format(sys.argv[0]))
+        exit(1)
+
+    res = bench_block_job(sys.argv[1], json.loads(sys.argv[2]), sys.argv[3:])
+    if 'seconds' in res:
+        print('{:.2f}'.format(res['seconds']))
+    else:
+        print(res)
--
2.18.0


Reply | Threaded
Open this post in threaded view
|

[PATCH for-5.0 v2 3/3] python: add example usage of simplebench

Vladimir Sementsov-Ogievskiy-2
In reply to this post by Vladimir Sementsov-Ogievskiy-2
This example may be used as a template for custom benchmark.
It illustrates three things to prepare:
 - define bench_func
 - define test environments (columns)
 - define test cases (rows)
And final call of simplebench API.

Signed-off-by: Vladimir Sementsov-Ogievskiy <[hidden email]>
---
 python/bench-example.py | 80 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100644 python/bench-example.py

diff --git a/python/bench-example.py b/python/bench-example.py
new file mode 100644
index 0000000000..4ccd4fc3dd
--- /dev/null
+++ b/python/bench-example.py
@@ -0,0 +1,80 @@
+#!/usr/bin/env python3
+#
+# Benchmark example
+#
+# Copyright (c) 2019 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import simplebench
+from qemu.bench_block_job import bench_block_copy, drv_file, drv_nbd
+
+
+def bench_func(env, case):
+    """ Handle one "cell" of benchmarking table. """
+    return bench_block_copy(env['qemu_binary'], env['cmd'],
+                            case['source'], case['target'])
+
+
+# You may set the following five variables to correct values, to turn this
+# example to real benchmark.
+ssd_source = '/path-to-raw-source-image-at-ssd'
+ssd_target = '/path-to-raw-target-image-at-ssd'
+hdd_target = '/path-to-raw-source-image-at-hdd'
+nbd_ip = 'nbd-ip-addr'
+nbd_port = 'nbd-port-number'
+
+# Test-cases are "rows" in benchmark resulting table, 'id' is a caption for
+# the row, other fields are handled by bench_func.
+test_cases = [
+    {
+        'id': 'ssd -> ssd',
+        'source': drv_file(ssd_source),
+        'target': drv_file(ssd_target)
+    },
+    {
+        'id': 'ssd -> hdd',
+        'source': drv_file(ssd_source),
+        'target': drv_file(hdd_target)
+    },
+    {
+        'id': 'ssd -> nbd',
+        'source': drv_file(ssd_source),
+        'target': drv_nbd(nbd_ip, nbd_port)
+    },
+]
+
+# Test-envs are "columns" in benchmark resulting table, 'id is a caption for
+# the column, other fields are handled by bench_func.
+test_envs = [
+    {
+        'id': 'backup-1',
+        'cmd': 'blockdev-backup',
+        'qemu_binary': '/path-to-qemu-binary-1'
+    },
+    {
+        'id': 'backup-2',
+        'cmd': 'blockdev-backup',
+        'qemu_binary': '/path-to-qemu-binary-2'
+    },
+    {
+        'id': 'mirror',
+        'cmd': 'blockdev-mirror',
+        'qemu_binary': '/path-to-qemu-binary-1'
+    }
+]
+
+result = simplebench.bench(bench_func, test_envs, test_cases, count=3)
+print(simplebench.ascii(result))
--
2.18.0


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH for-5.0 v2 0/3] benchmark util

Aleksandar Markovic-5
In reply to this post by Vladimir Sementsov-Ogievskiy-2
On Tue, Nov 26, 2019 at 4:49 PM Vladimir Sementsov-Ogievskiy
<[hidden email]> wrote:

>
> Hi all!
>
> Here is simple benchmarking utility, to generate performance
> comparison tables, like the following:
>
> ----------  -------------  -------------  -------------
>             backup-1       backup-2       mirror
> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
> ----------  -------------  -------------  -------------
>
> This is a v2, as v1 was inside
>  "[RFC 00/24] backup performance: block_status + async"
>
> I'll use this benchmark in other series, hope someone
> will like it.
>

Vladimir,

I really like this idea, even though I am interested in benchmarks of
different nature. Certainly a beautiful and handy tool for doing
detection of performance regressions (and also confirmation of
performance optimizations).

Did you run the tool on the previous QEMU versions, to detect change
in numbers between QEMU versions? Do you have the results of the same
benchmark for QEMU 2.12, 3.0, 4.0,... ?

What are units used in the table? Seconds, minutes? Hopefully, not hours?

Yours,
Aleksandar

> Vladimir Sementsov-Ogievskiy (3):
>   python: add simplebench.py
>   python: add qemu/bench_block_job.py
>   python: add example usage of simplebench
>
>  python/bench-example.py        |  80 +++++++++++++++++++++
>  python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>  python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>  3 files changed, 323 insertions(+)
>  create mode 100644 python/bench-example.py
>  create mode 100755 python/qemu/bench_block_job.py
>  create mode 100644 python/simplebench.py
>
> --
> 2.18.0
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH for-5.0 v2 0/3] benchmark util

Vladimir Sementsov-Ogievskiy-2
27.11.2019 22:58, Aleksandar Markovic wrote:

> On Tue, Nov 26, 2019 at 4:49 PM Vladimir Sementsov-Ogievskiy
> <[hidden email]> wrote:
>>
>> Hi all!
>>
>> Here is simple benchmarking utility, to generate performance
>> comparison tables, like the following:
>>
>> ----------  -------------  -------------  -------------
>>              backup-1       backup-2       mirror
>> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
>> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
>> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
>> ----------  -------------  -------------  -------------
>>
>> This is a v2, as v1 was inside
>>   "[RFC 00/24] backup performance: block_status + async"
>>
>> I'll use this benchmark in other series, hope someone
>> will like it.
>>
>
> Vladimir,
>
> I really like this idea, even though I am interested in benchmarks of
> different nature. Certainly a beautiful and handy tool for doing
> detection of performance regressions (and also confirmation of
> performance optimizations).

Hi Aleksandar!

Glad you are interested!

>
> Did you run the tool on the previous QEMU versions, to detect change
> in numbers between QEMU versions? Do you have the results of the same
> benchmark for QEMU 2.12, 3.0, 4.0,... ?

No, I don't really interested in such comparisons, my goal is to measure
backup performance in context of my backup/block-copy series.

But such statistics may be simply generated, just make several git work-trees
per release you interested in, fill test_envs variable (together with other
variables) in python/bench-example.py (patch 03), and run it.

>
> What are units used in the table? Seconds, minutes? Hopefully, not hours?

Seconds of course, I'm not so patient :)
Also, if you are interested, 1000M raw images filled with ones used to make
this benchmark table. However, it's in cover-letter only as example.

backup-1 and mirror are upstream. backup-2 is our downstream. nbd gain of
backup-2 is related to async requests. ssd gain for backup-1 is related to
copy offloading (wow, it works!).

>
> Yours,
> Aleksandar
>
>> Vladimir Sementsov-Ogievskiy (3):
>>    python: add simplebench.py
>>    python: add qemu/bench_block_job.py
>>    python: add example usage of simplebench
>>
>>   python/bench-example.py        |  80 +++++++++++++++++++++
>>   python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>>   python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>>   3 files changed, 323 insertions(+)
>>   create mode 100644 python/bench-example.py
>>   create mode 100755 python/qemu/bench_block_job.py
>>   create mode 100644 python/simplebench.py
>>
>> --
>> 2.18.0
>>
>>


--
Best regards,
Vladimir
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH for-5.0 v2 0/3] benchmark util

Cleber Rosa
In reply to this post by Vladimir Sementsov-Ogievskiy-2
On Tue, Nov 26, 2019 at 06:48:45PM +0300, Vladimir Sementsov-Ogievskiy wrote:

> Hi all!
>
> Here is simple benchmarking utility, to generate performance
> comparison tables, like the following:
>
> ----------  -------------  -------------  -------------
>             backup-1       backup-2       mirror
> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
> ----------  -------------  -------------  -------------
>
> This is a v2, as v1 was inside
>  "[RFC 00/24] backup performance: block_status + async"
>
> I'll use this benchmark in other series, hope someone
> will like it.
>
> Vladimir Sementsov-Ogievskiy (3):
>   python: add simplebench.py
>   python: add qemu/bench_block_job.py
>   python: add example usage of simplebench
>
>  python/bench-example.py        |  80 +++++++++++++++++++++
>  python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>  python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>  3 files changed, 323 insertions(+)
>  create mode 100644 python/bench-example.py
>  create mode 100755 python/qemu/bench_block_job.py
>  create mode 100644 python/simplebench.py
>
> --
> 2.18.0
>
Hi Vladimir,

This looks interesting.

Do you think the execution of "test cases" in an "environment" are a
generic enough concept that could be reused (or reuse other system)?
My point is that it'd be nice to do the same thing say for the
acceptance tests, or any tests for that matter.  For instance, for
known parameters, we could record what's the time difference between
booting a guest with q35 or pc machine types and virtio-block or
virtio-scsi devices.

BTW, This reminded me of a IOzone[1] test runner / results analyzer:

  https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/io/disk/iozone.py

I'm also cc'ing Lukáš Doktor, who has actively worked in something
similar.

Cheers,
- Cleber.

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH for-5.0 v2 0/3] benchmark util

Vladimir Sementsov-Ogievskiy-2
06.12.2019 22:43, Cleber Rosa wrote:

> On Tue, Nov 26, 2019 at 06:48:45PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all!
>>
>> Here is simple benchmarking utility, to generate performance
>> comparison tables, like the following:
>>
>> ----------  -------------  -------------  -------------
>>              backup-1       backup-2       mirror
>> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
>> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
>> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
>> ----------  -------------  -------------  -------------
>>
>> This is a v2, as v1 was inside
>>   "[RFC 00/24] backup performance: block_status + async"
>>
>> I'll use this benchmark in other series, hope someone
>> will like it.
>>
>> Vladimir Sementsov-Ogievskiy (3):
>>    python: add simplebench.py
>>    python: add qemu/bench_block_job.py
>>    python: add example usage of simplebench
>>
>>   python/bench-example.py        |  80 +++++++++++++++++++++
>>   python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>>   python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>>   3 files changed, 323 insertions(+)
>>   create mode 100644 python/bench-example.py
>>   create mode 100755 python/qemu/bench_block_job.py
>>   create mode 100644 python/simplebench.py
>>
>> --
>> 2.18.0
>>
>
> Hi Vladimir,
>
> This looks interesting.
>
> Do you think the execution of "test cases" in an "environment" are a
> generic enough concept that could be reused (or reuse other system)?

that was my goal: to make something generic enough, but simple. Yes,
it can be reused for anything

> My point is that it'd be nice to do the same thing say for the
> acceptance tests, or any tests for that matter.  For instance, for
> known parameters, we could record what's the time difference between
> booting a guest with q35 or pc machine types and virtio-block or
> virtio-scsi devices.
>
> BTW, This reminded me of a IOzone[1] test runner / results analyzer:
>
>    https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/io/disk/iozone.py
>
> I'm also cc'ing Lukáš Doktor, who has actively worked in something
> similar.
>
> Cheers,
> - Cleber.
>


--
Best regards,
Vladimir
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH for-5.0 v2 0/3] benchmark util

Vladimir Sementsov-Ogievskiy-2
In reply to this post by Vladimir Sementsov-Ogievskiy-2
ping

26.11.2019 18:48, Vladimir Sementsov-Ogievskiy wrote:

> Hi all!
>
> Here is simple benchmarking utility, to generate performance
> comparison tables, like the following:
>
> ----------  -------------  -------------  -------------
>              backup-1       backup-2       mirror
> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
> ----------  -------------  -------------  -------------
>
> This is a v2, as v1 was inside
>   "[RFC 00/24] backup performance: block_status + async"
>
> I'll use this benchmark in other series, hope someone
> will like it.
>
> Vladimir Sementsov-Ogievskiy (3):
>    python: add simplebench.py
>    python: add qemu/bench_block_job.py
>    python: add example usage of simplebench
>
>   python/bench-example.py        |  80 +++++++++++++++++++++
>   python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>   python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>   3 files changed, 323 insertions(+)
>   create mode 100644 python/bench-example.py
>   create mode 100755 python/qemu/bench_block_job.py
>   create mode 100644 python/simplebench.py
>


--
Best regards,
Vladimir
Reply | Threaded
Open this post in threaded view
|

ping Re: [PATCH for-5.0 v2 0/3] benchmark util

Vladimir Sementsov-Ogievskiy-2
In reply to this post by Vladimir Sementsov-Ogievskiy-2
ping

26.11.2019 18:48, Vladimir Sementsov-Ogievskiy wrote:

> Hi all!
>
> Here is simple benchmarking utility, to generate performance
> comparison tables, like the following:
>
> ----------  -------------  -------------  -------------
>              backup-1       backup-2       mirror
> ssd -> ssd  0.43 +- 0.00   4.48 +- 0.06   4.38 +- 0.02
> ssd -> hdd  10.60 +- 0.08  10.69 +- 0.18  10.57 +- 0.05
> ssd -> nbd  33.81 +- 0.37  10.67 +- 0.17  10.07 +- 0.07
> ----------  -------------  -------------  -------------
>
> This is a v2, as v1 was inside
>   "[RFC 00/24] backup performance: block_status + async"
>
> I'll use this benchmark in other series, hope someone
> will like it.
>
> Vladimir Sementsov-Ogievskiy (3):
>    python: add simplebench.py
>    python: add qemu/bench_block_job.py
>    python: add example usage of simplebench
>
>   python/bench-example.py        |  80 +++++++++++++++++++++
>   python/qemu/bench_block_job.py | 115 +++++++++++++++++++++++++++++
>   python/simplebench.py          | 128 +++++++++++++++++++++++++++++++++
>   3 files changed, 323 insertions(+)
>   create mode 100644 python/bench-example.py
>   create mode 100755 python/qemu/bench_block_job.py
>   create mode 100644 python/simplebench.py
>


--
Best regards,
Vladimir