- 大数据集群压力测试选用hive-testbench生成测试数据。
- 使用Hive-testbench内置的分析TPC-DS分析模型进行复杂sql语句查询
- 测试3T数据量运行情况
- 预备阶段：
- 上传编译好的Hive-testbench压缩包
- sudo tar -vxf hive-testbench.tar.gz
- cd hive-testbench
- 一. 生产3T测试数据压力测试
- 生成测试数据之前集群各DataNode存储情况
- su hive
- ./tpcds-setup.sh 3000
- 生成3T测试数据后
- 生成测试数据之前集群各DataNode存储情况
- 二. 运行复杂sql
- select i_item_id,
- avg(ss_quantity) agg1,
- avg(ss_list_price) agg2,
- avg(ss_coupon_amt) agg3,
- avg(ss_sales_price) agg4
- from store_sales, customer_demographics, date_dim, item, promotion
- where ss_sold_date_sk = d_date_sk and
- ss_item_sk = i_item_sk and
- ss_cdemo_sk = cd_demo_sk and
- ss_promo_sk = p_promo_sk and
- cd_gender = ‘F’ and
- cd_marital_status = ‘W’ and
- cd_education_status = ‘College’ and
- (p_channel_email = ‘N’ or p_channel_event = ‘N’) and
- d_year = 2001
- group by i_item_id
- order by i_item_id
- limit 100;
- 三. 运行结果
作者 admin
张宴银，大数据开发工程师