gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
Page: Benchmarks from jinleileiking
Pages
AWS CLI with SeaweedFS AWS IAM CLI Actual Users Amazon IAM API Amazon S3 API Applications Async Backup Async Filer Metadata Backup Async Replication to Cloud Async Replication to another Filer Benchmark SeaweedFS as a GlusterFS replacement Benchmarks from jinleileiking Benchmarks Cache Remote Storage Choosing a Filer Store Client Libraries Cloud Drive Architecture Cloud Drive Benefits Cloud Drive Quick Setup Cloud Monitoring Cloud Tier Components Configure Remote Storage Customize Filer Store Data Backup Data Structure for Large Files Deployment to Kubernetes and Minikube Directories and Files Docker Image Registry with SeaweedFS Environment Variables Erasure Coding for warm storage FAQ FIO benchmark FUSE Mount Failover Master Server Filer Active Active cross cluster continuous synchronization Filer Cassandra Setup Filer Change Data Capture Filer Commands and Operations Filer Data Encryption Filer JWT Use Filer Metadata Events Filer Redis Setup Filer Server API Filer Setup Filer Store Replication Filer Stores Filer as a Key Large Value Store Gateway to Remote Object Storage Getting Started HDFS via S3 connector Hadoop Benchmark Hadoop Compatible File System Hardware Home Independent Benchmarks Kubernetes Backups and Recovery with K8up Large File Handling Load Command Line Options from a file Master Server API Migrate to Filer Store Mount Remote Storage Optimization Path Specific Configuration Path Specific Filer Store Production Setup Replication Run Blob Storage on Public Internet Run Presto on SeaweedFS S3 API Audit log S3 API Benchmark S3 API FAQ S3 Bucket Quota S3 Nginx Proxy SRV Service Discovery SeaweedFS Java Client SeaweedFS in Docker Swarm Security Configuration Security Overview Server Startup Setup Store file with a Time To Live Super Large Directories System Metrics TensorFlow with SeaweedFS Tiered Storage UrBackup with SeaweedFS Use Cases Volume Management Volume Server API WebDAV Words from SeaweedFS Users nodejs with Seaweed S3 rclone with SeaweedFS restic with SeaweedFS run HBase on SeaweedFS run Spark on SeaweedFS s3cmd with SeaweedFS weed shell
16 Benchmarks from jinleileiking
jinleileiking edited this page 2017-07-08 08:22:40 +08:00
Unescape Escape
Table of Contents
This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

1 region

env:

  • 2 pc E5_2630@2.30GHz * 2
    • DELL_SAS2.5_600G * 2
    • RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2208 [Thunderbolt] (rev 05)
      • raid 10
    • ext3
    • 128G mem
  • run benchmark from 1pc
  • fio 顺序写 iops 18k fio -filename=./a -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=16k -size=1G -numjobs=30 -runtime=1000 -group_reporting -name=mytest
  • 随机写2k
  • 添加preallocate，无优化。I0708 08:14:27 29165 volume_create_linux.go:16] Preallocated 31457280000 bytes disk space for data/benchmark_2.dat
  • 启动命令 ./weed server -master.peers "10.64.7.106:9666,10.4.23.114:9666,10.4.23.115:9666" -ip 10.4.23.114 -ip.bind 10.4.23.114 -master.port 9666 -dir ./data -volume.port 9555 -master.volumeSizeLimitMB 60000 -master.volumePreallocate

测试命令

修改了源代码，保证每个master只有一个volume

./weed benchmark -server 10.4.23.115:9666 -size 1000000 -n 40000

测试结果:

This is SeaweedFS version 0.76 linux amd64

------------ Writing Benchmark ----------
Completed 984 of 40000 requests, 2.5% 983.8/s 938.2MB/s
Completed 2026 of 40000 requests, 5.1% 1042.2/s 993.9MB/s
Completed 2843 of 40000 requests, 7.1% 817.0/s 779.2MB/s
Completed 3916 of 40000 requests, 9.8% 1073.0/s 1023.3MB/s
Completed 4970 of 40000 requests, 12.4% 1054.0/s 1005.2MB/s
Completed 5998 of 40000 requests, 15.0% 1028.0/s 980.4MB/s
Completed 7042 of 40000 requests, 17.6% 1044.0/s 995.7MB/s
Completed 8037 of 40000 requests, 20.1% 995.0/s 948.9MB/s
Completed 9068 of 40000 requests, 22.7% 1031.0/s 983.3MB/s
Completed 10097 of 40000 requests, 25.2% 1029.0/s 981.4MB/s
Completed 11150 of 40000 requests, 27.9% 1053.0/s 1004.3MB/s
Completed 12201 of 40000 requests, 30.5% 1051.0/s 1002.3MB/s
Completed 13229 of 40000 requests, 33.1% 1028.0/s 980.4MB/s
Completed 14269 of 40000 requests, 35.7% 1040.0/s 991.8MB/s
Completed 15281 of 40000 requests, 38.2% 1012.0/s 965.2MB/s
Completed 16298 of 40000 requests, 40.7% 1017.0/s 969.9MB/s
Completed 17353 of 40000 requests, 43.4% 1055.0/s 1006.2MB/s
Completed 18353 of 40000 requests, 45.9% 1000.0/s 953.7MB/s
Completed 18995 of 40000 requests, 47.5% 642.0/s 612.3MB/s
Completed 19988 of 40000 requests, 50.0% 992.9/s 946.9MB/s
Completed 21031 of 40000 requests, 52.6% 1043.1/s 994.8MB/s
Completed 22083 of 40000 requests, 55.2% 1052.0/s 1003.3MB/s
Completed 23104 of 40000 requests, 57.8% 1021.0/s 973.7MB/s
Completed 24160 of 40000 requests, 60.4% 1056.0/s 1007.1MB/s
Completed 25176 of 40000 requests, 62.9% 1016.0/s 969.0MB/s
Completed 26198 of 40000 requests, 65.5% 1022.0/s 974.7MB/s
Completed 27270 of 40000 requests, 68.2% 1072.0/s 1022.4MB/s
Completed 28279 of 40000 requests, 70.7% 1009.0/s 962.3MB/s
Completed 28776 of 40000 requests, 71.9% 497.0/s 474.0MB/s
Completed 28776 of 40000 requests, 71.9% 0.0/s 0.0MB/s
Completed 28776 of 40000 requests, 71.9% 0.0/s 0.0MB/s
Completed 28776 of 40000 requests, 71.9% 0.0/s 0.0MB/s
Completed 28958 of 40000 requests, 72.4% 182.0/s 173.6MB/s
Completed 29457 of 40000 requests, 73.6% 499.0/s 475.9MB/s
Completed 29590 of 40000 requests, 74.0% 133.0/s 126.8MB/s
Completed 29702 of 40000 requests, 74.3% 112.0/s 106.8MB/s
Completed 29819 of 40000 requests, 74.5% 117.0/s 111.6MB/s
Completed 29936 of 40000 requests, 74.8% 117.0/s 111.6MB/s
Completed 30109 of 40000 requests, 75.3% 173.0/s 165.0MB/s
Completed 30250 of 40000 requests, 75.6% 141.0/s 134.5MB/s
Completed 30387 of 40000 requests, 76.0% 137.0/s 130.7MB/s
Completed 30483 of 40000 requests, 76.2% 96.0/s 91.6MB/s
Completed 30616 of 40000 requests, 76.5% 133.0/s 126.8MB/s
Completed 30718 of 40000 requests, 76.8% 102.0/s 97.3MB/s
Completed 30820 of 40000 requests, 77.0% 102.0/s 97.3MB/s
Completed 30967 of 40000 requests, 77.4% 147.0/s 140.2MB/s
Completed 31096 of 40000 requests, 77.7% 129.0/s 123.0MB/s
Completed 31223 of 40000 requests, 78.1% 127.0/s 121.1MB/s
Completed 31398 of 40000 requests, 78.5% 175.0/s 166.9MB/s
Completed 31505 of 40000 requests, 78.8% 107.0/s 102.0MB/s
Completed 31631 of 40000 requests, 79.1% 126.0/s 120.2MB/s
Completed 31736 of 40000 requests, 79.3% 105.0/s 100.1MB/s
Completed 31878 of 40000 requests, 79.7% 142.0/s 135.4MB/s
Completed 32033 of 40000 requests, 80.1% 155.0/s 147.8MB/s
Completed 32141 of 40000 requests, 80.4% 108.0/s 103.0MB/s
Completed 32289 of 40000 requests, 80.7% 148.0/s 141.2MB/s
Completed 32397 of 40000 requests, 81.0% 108.0/s 103.0MB/s
Completed 32546 of 40000 requests, 81.4% 149.0/s 142.1MB/s
Completed 32645 of 40000 requests, 81.6% 99.0/s 94.4MB/s
Completed 32776 of 40000 requests, 81.9% 131.0/s 124.9MB/s
Completed 32933 of 40000 requests, 82.3% 157.0/s 149.7MB/s
Completed 33029 of 40000 requests, 82.6% 96.0/s 91.6MB/s
Completed 33139 of 40000 requests, 82.8% 110.0/s 104.9MB/s
Completed 33292 of 40000 requests, 83.2% 153.0/s 145.9MB/s
Completed 33417 of 40000 requests, 83.5% 125.0/s 119.2MB/s
Completed 33540 of 40000 requests, 83.8% 123.0/s 117.3MB/s
Completed 33694 of 40000 requests, 84.2% 154.0/s 146.9MB/s
Completed 33850 of 40000 requests, 84.6% 156.0/s 148.8MB/s
Completed 34006 of 40000 requests, 85.0% 156.0/s 148.8MB/s
Completed 34176 of 40000 requests, 85.4% 170.0/s 162.1MB/s
Completed 34303 of 40000 requests, 85.8% 127.0/s 121.1MB/s
Completed 34491 of 40000 requests, 86.2% 188.0/s 179.3MB/s
Completed 34623 of 40000 requests, 86.6% 132.0/s 125.9MB/s
Completed 34818 of 40000 requests, 87.0% 195.0/s 186.0MB/s
Completed 34978 of 40000 requests, 87.4% 160.0/s 152.6MB/s
Completed 35141 of 40000 requests, 87.9% 163.0/s 155.4MB/s
Completed 35297 of 40000 requests, 88.2% 156.0/s 148.8MB/s
Completed 35430 of 40000 requests, 88.6% 133.0/s 126.8MB/s
Completed 35634 of 40000 requests, 89.1% 204.0/s 194.6MB/s
Completed 35757 of 40000 requests, 89.4% 123.0/s 117.3MB/s
Completed 35913 of 40000 requests, 89.8% 156.0/s 148.8MB/s
Completed 36042 of 40000 requests, 90.1% 129.0/s 123.0MB/s
Completed 36151 of 40000 requests, 90.4% 109.0/s 104.0MB/s
Completed 36299 of 40000 requests, 90.7% 148.0/s 141.1MB/s
Completed 36429 of 40000 requests, 91.1% 130.0/s 124.0MB/s
Completed 36549 of 40000 requests, 91.4% 120.0/s 114.4MB/s
Completed 36683 of 40000 requests, 91.7% 134.0/s 127.8MB/s
Completed 36769 of 40000 requests, 91.9% 86.0/s 82.0MB/s
Completed 36878 of 40000 requests, 92.2% 109.0/s 104.0MB/s
Completed 36983 of 40000 requests, 92.5% 105.0/s 100.1MB/s
Completed 37105 of 40000 requests, 92.8% 122.0/s 116.4MB/s
Completed 37242 of 40000 requests, 93.1% 137.0/s 130.6MB/s
Completed 37360 of 40000 requests, 93.4% 118.0/s 112.5MB/s
Completed 37500 of 40000 requests, 93.8% 140.0/s 133.5MB/s
Completed 37606 of 40000 requests, 94.0% 106.0/s 101.1MB/s
Completed 37742 of 40000 requests, 94.4% 136.0/s 129.7MB/s
Completed 37879 of 40000 requests, 94.7% 137.0/s 130.7MB/s
Completed 38010 of 40000 requests, 95.0% 131.0/s 124.9MB/s
Completed 38105 of 40000 requests, 95.3% 95.0/s 90.6MB/s
Completed 38198 of 40000 requests, 95.5% 93.0/s 88.7MB/s
Completed 38345 of 40000 requests, 95.9% 147.0/s 140.2MB/s
Completed 38457 of 40000 requests, 96.1% 112.0/s 106.8MB/s
Completed 38582 of 40000 requests, 96.5% 125.0/s 119.2MB/s
Completed 38697 of 40000 requests, 96.7% 115.0/s 109.7MB/s
Completed 38833 of 40000 requests, 97.1% 136.0/s 129.7MB/s
Completed 38921 of 40000 requests, 97.3% 88.0/s 83.9MB/s
Completed 38992 of 40000 requests, 97.5% 71.0/s 67.7MB/s
Completed 39057 of 40000 requests, 97.6% 65.0/s 62.0MB/s
Completed 39170 of 40000 requests, 97.9% 113.0/s 107.8MB/s
Completed 39263 of 40000 requests, 98.2% 93.0/s 88.7MB/s
Completed 39329 of 40000 requests, 98.3% 66.0/s 62.9MB/s
Completed 39400 of 40000 requests, 98.5% 71.0/s 67.7MB/s
Completed 39477 of 40000 requests, 98.7% 77.0/s 73.4MB/s
Completed 39584 of 40000 requests, 99.0% 107.0/s 102.0MB/s
Completed 39716 of 40000 requests, 99.3% 132.0/s 125.9MB/s
Completed 39796 of 40000 requests, 99.5% 80.0/s 76.3MB/s
Completed 39965 of 40000 requests, 99.9% 169.0/s 161.2MB/s

Concurrency Level:      16
Time taken for tests:   117.609 seconds
Complete requests:      40000
Failed requests:        0
Total transferred:      40001250749 bytes
Requests per second:    340.11 [#/sec]
Transfer rate:          332149.44 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        7.0      47.0       4373.3      125.6

Percentage of the requests served within a certain time (ms)
   50%     14.8 ms
   66%     17.9 ms
   75%     20.0 ms
   80%     21.7 ms
   90%    150.4 ms
   95%    267.8 ms
   98%    366.6 ms
   99%    445.9 ms
  100%    4373.3 ms

------------ Randomly Reading Benchmark ----------
Completed 2228 of 40000 requests, 5.6% 2227.8/s 2124.7MB/s
Completed 4540 of 40000 requests, 11.3% 2312.0/s 2205.0MB/s
Completed 6790 of 40000 requests, 17.0% 2250.0/s 2145.8MB/s
Completed 9062 of 40000 requests, 22.7% 2272.0/s 2166.8MB/s
Completed 11370 of 40000 requests, 28.4% 2308.0/s 2201.1MB/s
Completed 13691 of 40000 requests, 34.2% 2321.0/s 2213.6MB/s
Completed 15972 of 40000 requests, 39.9% 2280.2/s 2174.6MB/s
Completed 18295 of 40000 requests, 45.7% 2323.6/s 2216.1MB/s
Completed 20611 of 40000 requests, 51.5% 2316.2/s 2209.0MB/s
Completed 22949 of 40000 requests, 57.4% 2338.0/s 2229.8MB/s
Completed 25243 of 40000 requests, 63.1% 2294.0/s 2187.8MB/s
Completed 27550 of 40000 requests, 68.9% 2307.0/s 2200.2MB/s
Completed 29881 of 40000 requests, 74.7% 2331.0/s 2223.1MB/s
Completed 32217 of 40000 requests, 80.5% 2336.0/s 2227.8MB/s
Completed 34514 of 40000 requests, 86.3% 2297.0/s 2190.7MB/s
Completed 36826 of 40000 requests, 92.1% 2311.9/s 2204.9MB/s
Completed 39189 of 40000 requests, 98.0% 2363.0/s 2253.6MB/s

Concurrency Level:      16
Time taken for tests:   17.349 seconds
Complete requests:      40000
Failed requests:        0
Total transferred:      40001249878 bytes
Requests per second:    2305.66 [#/sec]
Transfer rate:          2251686.93 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        1.1      6.8       46.7      4.6

Percentage of the requests served within a certain time (ms)
   50%      5.6 ms
   66%      8.0 ms
   75%      9.7 ms
   80%     10.8 ms
   90%     13.4 ms
   95%     15.5 ms
   98%     18.2 ms
   99%     20.4 ms
  100%     46.7 ms

测试结束后，ioutil还是100，写iops还是很大，iotop如下

 be/4 root        0.00 B/s    0.00 B/s  0.00 % 99.99 % [flush-8:0]
  663 be/4 root        0.00 B/s    0.00 B/s  0.00 % 99.99 % [kjournald]

历史测试

./weed benchmark -server 10.4.23.115:9666 -n 10000 -size 1000000 -c 100

Concurrency Level:      100
Time taken for tests:   6.808 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10000315338 bytes
Requests per second:    1468.89 [#/sec]
Transfer rate:          1434508.39 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        7.5      67.6       730.1      66.6

Percentage of the requests served within a certain time (ms)
   50%     51.0 ms
   66%     72.0 ms
   75%     84.5 ms
   80%     93.2 ms
   90%    126.8 ms
   95%    208.6 ms
   98%    286.5 ms
   99%    345.1 ms
  100%    730.1 ms

./weed benchmark -server 10.4.23.115:9666 -n 10000 -size 1000000

Concurrency Level:      16
Time taken for tests:   8.523 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10000317165 bytes
Requests per second:    1173.36 [#/sec]
Transfer rate:          1145892.52 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        7.7      13.6       49.1      2.9

Percentage of the requests served within a certain time (ms)
   50%     13.2 ms
   66%     14.2 ms
   75%     14.9 ms
   80%     15.4 ms
   90%     17.0 ms
   95%     18.5 ms
   98%     20.6 ms
   99%     22.3 ms
  100%     49.1 ms

2 region

env:

  • 3 pc E5_2630@2.30GHz * 2
    • DELL_SAS2.5_600G * 2
    • RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2208 [Thunderbolt] (rev 05)
      • raid 10
    • 128G mem
    • 2 data region
  • run benchmark from 1pc ( I have not got the 4)
  • cmdweed benchmark -server 10.64.7.106:9666 -size 1000000 -n 10000 rep:000

write

Concurrency Level:      16
Time taken for tests:   141.200 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10000313420 bytes
Requests per second:    70.82 [#/sec]
Transfer rate:          69163.65 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        7.8      225.6       1986.8      207.2

Percentage of the requests served within a certain time (ms)
   50%    198.9 ms
   66%    281.8 ms
   75%    337.8 ms
   80%    367.9 ms
   90%    476.7 ms
   95%    588.3 ms
   98%    763.7 ms
   99%    892.9 ms
  100%    1986.8 ms

read

Concurrency Level:      16
Time taken for tests:   59.509 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10000311600 bytes
Requests per second:    168.04 [#/sec]
Transfer rate:          164107.92 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        1.2      94.0       2507.3      152.9

Percentage of the requests served within a certain time (ms)
   50%     75.5 ms
   66%     90.7 ms
   75%     98.3 ms
   80%    102.9 ms
   90%    140.9 ms
   95%    344.9 ms
   98%    639.8 ms
   99%    834.7 ms
  100%    2507.3 ms

conconrrent to 100

weed benchmark -server 10.64.7.106:9666 -size 1000000 -n 10000 -c 100

Concurrency Level:      100
Time taken for tests:   61.658 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10000313519 bytes
Requests per second:    162.18 [#/sec]
Transfer rate:          158388.25 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        8.0      606.1       3317.2      720.7

1k test

weed benchmark -server 10.64.7.106:9666 -n 10000 -c 100

------------ Writing Benchmark ----------
Completed 7574 of 10000 requests, 75.7% 7573.6/s 7.6MB/s

Concurrency Level:      100
Time taken for tests:   1.327 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      10552563 bytes
Requests per second:    7534.50 [#/sec]
Transfer rate:          7764.48 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.3      13.0       329.9      14.8

Percentage of the requests served within a certain time (ms)
   50%      1.1 ms
   66%     27.7 ms
   75%     27.8 ms
   90%     28.2 ms
   95%     28.7 ms
   98%     29.6 ms
   99%     55.0 ms
  100%    329.9 ms

Introduction

API

Configuration

Filer

Filer Stores

Advanced Filer Configurations

FUSE Mount

WebDAV

Cloud Drive

AWS S3 API

AWS IAM

Machine Learning

HDFS

Replication and Backup

Messaging

Use Cases

Operations

Advanced

Security

Misc Use Case Examples