Rally で OpenStack のベンチマークを取る

Rally

OpenStack のベンチマークを取ってくれるツール。ドキュメントの図を見ると以下の事が出来るらしい。

  • Deploy
    • Rally を使って OpenStack をデプロイ出来る。もちろん既存の環境を使用することも可能。
  • Verify
    • OpenStack 公式のテストツール tempest を使って OpenStack 環境をテストする。
  • Benchmark
  • Generate Report
    • グラフィカルなレポートを作成可能。

インストール

ドキュメントのInstallationを参照のこと。基本的には以下でインストール可能。

$ wget -q -O- https://raw.githubusercontent.com/openstack/rally/master/install_rally.sh | bash

既存 OpenStack 環境の登録

python の virtualenv 環境でインストールされるので、以下のコマンドを実行して環境を有効にする。

$ . ~/rally/bin/activate

現在の OpenStack 環境を Rally に登録する。ここでは、後のベンチマーク取得で既存ユーザーを使用するため、以下のような json ファイルを使用して環境を登録する。

{
    "type": "ExistingCloud",
    "auth_url": "http://example.net:5000/v2.0/",
    "region_name": "RegionOne",
    "endpoint_type": "public",
    "admin": {
        "username": "admin",
        "password": "pa55word",
        "tenant_name": "demo"
    },
    "users": [
        {
            "username": "b1",
            "password": "1234",
            "tenant_name": "testing"
        },
        {
            "username": "b2",
            "password": "1234",
            "tenant_name": "testing"
        }
    ]
}
$ rally deployment create --file ./existing_cloud.json --name home_cloud

これで、登録完了。

ベンチマーク取得

サンプルで用意されているシナリオファイルを使用してベンチマークを取る。以下の様なサンプルファイルが用意されている。

{
    "NovaServers.boot_and_delete_server": [
        {
            "args": {
                "flavor": {
                    "name": "m1.tiny"
                },
                "image": {
                    "name": "^cirros.*uec$"
                },
                "force_delete": false
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 2
            },
            "context": {
                "users": {
                    "tenants": 3,
                    "users_per_tenant": 2
                }
            }
        }
    ]
}

今回は既存ユーザーを使用してベンチマークを取るため、"context" 部分の "users" を削除する。 (デフォルトのサンプルのままだと、テナントの作成とユーザーの作成も行うみたい。。)

{
    "NovaServers.boot_and_delete_server": [
        {
            "args": {
                "flavor": {
                    "name": "m1.tiny"
                },
                "image": {
                    "name": "^cirros.*uec$"
                },
                "force_delete": false
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 2
            },
            "context": {
            }
        }
    ]
}

このファイルを使用してベンチマークを取得する。

$ rally task start ./boot-and-delete.json

$ rally task start ./boot-and-delete.json
--------------------------------------------------------------------------------
 Preparing input task
--------------------------------------------------------------------------------

Input task is:

{
    "NovaServers.boot_and_delete_server": [
        {
            "args": {
                "flavor": {
                    "name": "m1.tiny"
                },
                "image": {
                    "name": "cirros"
                },
                "force_delete": false
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 2
            },
            "context": {
            }
        }
    ]
}

--------------------------------------------------------------------------------
 Task  98241e54-4323-43fa-ad7a-a7cfaad34152: started
--------------------------------------------------------------------------------

Benchmarking... This can take a while...

To track task status use:

        rally task status
        or
        rally task detailed

Using task: 98241e54-4323-43fa-ad7a-a7cfaad34152
2015-08-31 14:30:36.466 2612 INFO rally.api [-] Benchmark Task 98241e54-4323-43fa-ad7a-a7cfaad34152 on Deployment 422e2e8e-5856-4b53-ba5d-aea068490bc2
2015-08-31 14:30:36.467 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Task validation.
2015-08-31 14:30:36.482 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Task validation of scenarios names.
2015-08-31 14:30:36.485 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Task validation of scenarios names.
2015-08-31 14:30:36.486 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Task validation of syntax.
2015-08-31 14:30:36.491 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Task validation of syntax.
2015-08-31 14:30:36.492 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Task validation of semantic.
2015-08-31 14:30:36.493 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Task validation check cloud.
2015-08-31 14:30:36.794 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Task validation check cloud.
2015-08-31 14:30:36.804 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Enter context: `existing_users`
2015-08-31 14:30:36.955 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Enter context: `existing_users`
2015-08-31 14:30:37.471 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Exit context: `existing_users`
2015-08-31 14:30:37.472 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Exit context: `existing_users`
2015-08-31 14:30:37.473 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Task validation of semantic.
2015-08-31 14:30:37.474 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Task validation.
2015-08-31 14:30:37.475 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Benchmarking.
2015-08-31 14:30:37.487 2612 INFO rally.task.engine [-] Running benchmark with key:
{
  "kw": {
    "runner": {
      "type": "constant",
      "concurrency": 2,
      "times": 10
    },
    "args": {
      "force_delete": false,
      "flavor": {
        "name": "m1.tiny"
      },
      "image": {
        "name": "cirros"
      }
    },
    "context": {}
  },
  "name": "NovaServers.boot_and_delete_server",
  "pos": 0
}
2015-08-31 14:30:37.496 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Enter context: `existing_users`
2015-08-31 14:30:37.645 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Enter context: `existing_users`
2015-08-31 14:30:38.012 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 0 START
2015-08-31 14:30:38.016 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 1 START
2015-08-31 14:31:06.429 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 0 END: OK
2015-08-31 14:31:06.440 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 2 START
2015-08-31 14:31:14.151 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 1 END: OK
2015-08-31 14:31:14.157 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 3 START
2015-08-31 14:31:28.191 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 2 END: OK
2015-08-31 14:31:28.195 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 4 START
2015-08-31 14:31:37.357 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 3 END: OK
2015-08-31 14:31:37.363 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 5 START
2015-08-31 14:31:50.829 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 4 END: OK
2015-08-31 14:31:50.838 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 6 START
2015-08-31 14:31:57.941 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 5 END: OK
2015-08-31 14:31:57.947 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 7 START
2015-08-31 14:32:12.279 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 6 END: OK
2015-08-31 14:32:12.285 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 8 START
2015-08-31 14:32:19.989 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 7 END: OK
2015-08-31 14:32:19.995 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 9 START
2015-08-31 14:32:35.312 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 8 END: OK
2015-08-31 14:32:39.725 2620 INFO rally.task.runner [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | ITER: 9 END: OK
2015-08-31 14:32:39.734 2612 INFO rally.plugins.openstack.context.cleanup.context [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  user resources cleanup
2015-08-31 14:32:40.758 2612 INFO rally.plugins.openstack.context.cleanup.context [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: user resources cleanup
2015-08-31 14:32:40.759 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Starting:  Exit context: `existing_users`
2015-08-31 14:32:40.760 2612 INFO rally.plugins.openstack.context.keystone.existing_users [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Exit context: `existing_users`
2015-08-31 14:32:40.862 2612 INFO rally.task.engine [-] Task 98241e54-4323-43fa-ad7a-a7cfaad34152 | Completed: Benchmarking.

--------------------------------------------------------------------------------
Task 98241e54-4323-43fa-ad7a-a7cfaad34152: finished
--------------------------------------------------------------------------------

test scenario NovaServers.boot_and_delete_server
args position 0
args values:
{
  "runner": {
    "type": "constant",
    "concurrency": 2,
    "times": 10
  },
  "args": {
    "force_delete": false,
    "flavor": {
      "name": "m1.tiny"
    },
    "image": {
      "name": "cirros"
    }
  },
  "context": {}
}
+--------------------------------------------------------------------------------------------+
|                                    Response Times (sec)                                    |
+--------------------+--------+--------+--------+--------+--------+--------+---------+-------+
| action             | min    | median | 90%ile | 95%ile | max    | avg    | success | count |
+--------------------+--------+--------+--------+--------+--------+--------+---------+-------+
| nova.boot_server   | 17.132 | 18.931 | 26.374 | 29.873 | 33.373 | 20.841 | 100.0%  | 10    |
| nova.delete_server | 2.597  | 2.742  | 3.589  | 4.278  | 4.968  | 3.053  | 100.0%  | 10    |
| total              | 19.729 | 22.336 | 29.186 | 32.66  | 36.134 | 23.894 | 100.0%  | 10    |
+--------------------+--------+--------+--------+--------+--------+--------+---------+-------+
Load duration: 121.739664793
Full duration: 123.26568222

HINTS:
* To plot HTML graphics with this data, run:
        rally task report 98241e54-4323-43fa-ad7a-a7cfaad34152 --out output.html

* To generate a JUnit report, run:
        rally task report 98241e54-4323-43fa-ad7a-a7cfaad34152 --junit --out output.xml

* To get raw JSON output of task results, run:
        rally task results 98241e54-4323-43fa-ad7a-a7cfaad34152

ベンチマークの結果を html ファイルで取得して、表示してみます。

$ rally task report 98241e54-4323-43fa-ad7a-a7cfaad34152 --out output.html

f:id:ryouta768:20150901000121p:plain

f:id:ryouta768:20150901000140p:plain

・・・

本当は ベンチマークではなくテストを行いたかったので、次回は Rally を使ったテストに挑戦してみます。