Compare commits
11 Commits
5c0fe99959
...
9d24297585
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9d24297585 | ||
|
|
36d9fb73c7 | ||
|
|
6e917c8336 | ||
|
|
1b1acf62c1 | ||
|
|
dfd439d7cc | ||
|
|
4fcfd17223 | ||
|
|
a33043cedd | ||
|
|
667bd4356f | ||
|
|
bebf31d676 | ||
|
|
c1a6dfe7cb | ||
|
|
b63202ccb6 |
15
README.md
15
README.md
@ -1,11 +1,24 @@
|
|||||||
# storm
|
# storm
|
||||||
|
|
||||||
#### 介绍
|
#### 介绍
|
||||||
A free and open source distributed realtime computation system
|
Apache Storm是一个免费的开源分布式实时计算系统。 Storm可以轻松地处理无限数据流, 并且实时处理Hadoop批处理任务。 可以和任何编程语言一起使用,并且使用起来非常有趣!
|
||||||
|
Storm有许多用例:实时分析,在线机器学习,连续计算,分布式RPC,ETL等等。 Storm是快速 的:一个基准计算每个节点每秒处理超过一百万个元组。 它具有可扩展性,容错性,可确保您的 数据得到处理,且易于设置和操作。
|
||||||
|
Storm集成了您已经使用的队列和数据库技术。 Storm拓扑消耗数据流,并以任意复杂的方式处理 这些流,然后重新分配计算的每个阶段之间的流
|
||||||
|
|
||||||
#### 软件架构
|
#### 软件架构
|
||||||
软件架构说明
|
软件架构说明
|
||||||
|
与Hadoop主从架构一样,Storm也采用Master/Slave体系结构,分布式计算由Nimbus和Supervisor两类服务进程实现,Nimbus进程运行在集群的主节点,负责任务的指派和分发,Supervisor运行在集群的从节点,负责执行任务的具体部分。
|
||||||
|
- Nimbus: Storm集群的Master节点,负责资源分配和任务调度,负责分发用户代码,指派给具体的Supervisor节点上的Worker节点,去运行Topology对应的组件(Spout/Bolt)的Task。
|
||||||
|
- Supervisor: Storm集群的从节点,负责接受Nimbus分配的任务,启动和停止属于自己管理的worker进程。通过Storm的配置文件中的supervisor.slots.ports配置项,可以指定在一个Supervisor上最大允许多少个Slot,每个Slot通过端口号来唯一标识,一个端口号对应一个Worker进程(如果该Worker进程被启动)。
|
||||||
|
- Worker: 负责运行具体处理组件逻辑的进程。Worker运行的任务类型只有两种,一种是Spout任务,一种是Bolt任务。
|
||||||
|
- Task: worker中每一个spout/bolt的线程称为一个task。同一个spout/bolt的task可能会共享一个物理线程,该线程称为executor。
|
||||||
|
- ZooKeeper: 用来协调Nimbus和Supervisor,如果Supervisor因故障出现问题而无法运行Topology,Nimbus会第一时间感知到,并重新分配Topology到其它可用的Supervisor上运行
|
||||||
|
|
||||||
|
#### ARM支持:
|
||||||
|
|
||||||
|
1. 移植指南:xxxx
|
||||||
|
2. 部署指南:https://support.huaweicloud.com/dpmg-apache-kunpengbds/kunpengstorm_04_0001.html
|
||||||
|
3. 调优指南:https://support.huaweicloud.com/tngg-kunpengbds/kunpengstormhdp_05_0002.html
|
||||||
|
|
||||||
#### 安装教程
|
#### 安装教程
|
||||||
|
|
||||||
|
|||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
apache-storm-1.2.4.tar.gzac
Normal file
BIN
apache-storm-1.2.4.tar.gzac
Normal file
Binary file not shown.
@ -11,6 +11,7 @@ StandardError=syslog
|
|||||||
SyslogIdentifier=storm-nimbus
|
SyslogIdentifier=storm-nimbus
|
||||||
SyslogLevel=debug
|
SyslogLevel=debug
|
||||||
SuccessExitStatus=20
|
SuccessExitStatus=20
|
||||||
|
SuccessExitStatus=143
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
|
|||||||
@ -11,6 +11,7 @@ StandardError=syslog
|
|||||||
SyslogIdentifier=storm-supervisor
|
SyslogIdentifier=storm-supervisor
|
||||||
SyslogLevel=debug
|
SyslogLevel=debug
|
||||||
SuccessExitStatus=20
|
SuccessExitStatus=20
|
||||||
|
SuccessExitStatus=143
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
|
|||||||
24
storm.spec
24
storm.spec
@ -4,8 +4,8 @@
|
|||||||
%define _log_dir %{_var}/log/storm
|
%define _log_dir %{_var}/log/storm
|
||||||
%define _data_dir %{_sharedstatedir}/storm
|
%define _data_dir %{_sharedstatedir}/storm
|
||||||
Name: storm
|
Name: storm
|
||||||
Version: 1.2.3
|
Version: 1.2.4
|
||||||
Release: 4
|
Release: 2
|
||||||
Summary: Apache Storm realtime computation system
|
Summary: Apache Storm realtime computation system
|
||||||
License: Apache-2.0
|
License: Apache-2.0
|
||||||
Url: http://storm.apache.org
|
Url: http://storm.apache.org
|
||||||
@ -16,9 +16,9 @@ Source3: %{name}-logviewer.service
|
|||||||
Source4: %{name}-ui.service
|
Source4: %{name}-ui.service
|
||||||
Source5: %{name}.logrotate
|
Source5: %{name}.logrotate
|
||||||
Source6: %{name}.conf
|
Source6: %{name}.conf
|
||||||
Source7: apache-storm-1.2.3.tar.gzaa
|
Source7: apache-storm-1.2.4.tar.gzaa
|
||||||
Source8: apache-storm-1.2.3.tar.gzab
|
Source8: apache-storm-1.2.4.tar.gzab
|
||||||
Source9: apache-storm-1.2.3.tar.gzac
|
Source9: apache-storm-1.2.4.tar.gzac
|
||||||
BuildRoot: %{_tmppath}/%{name}-%{version}-build
|
BuildRoot: %{_tmppath}/%{name}-%{version}-build
|
||||||
%{?systemd_requires}
|
%{?systemd_requires}
|
||||||
BuildRequires: systemd
|
BuildRequires: systemd
|
||||||
@ -88,14 +88,20 @@ done
|
|||||||
%dir %{_data_dir}
|
%dir %{_data_dir}
|
||||||
|
|
||||||
%changelog
|
%changelog
|
||||||
* Fri 20 Aug 2021 sunguoshuai <sunguoshuai@huawei.com> 1.2.3 - 4
|
* Wed Sep 28 2022 xiexing <xiexing4@hisilicon.com> - 1.2.4-2
|
||||||
|
- fix stop service return 143
|
||||||
|
|
||||||
|
* Wed Sep 01 2021 wangkai <wangkai385@huawei.com> - 1.2.4-1
|
||||||
|
- Upgrade to 1.2.4 for fix CVE-2021-38294-CVE-2021-40865
|
||||||
|
|
||||||
|
* Fri Aug 20 2021 sunguoshuai <sunguoshuai@huawei.com> - 1.2.3-4
|
||||||
- Fix runtime error and uninstall warning
|
- Fix runtime error and uninstall warning
|
||||||
|
|
||||||
* Mon Jun 14 2021 lingsheng <lingsheng@huawei.com> 1.2.3 - 3
|
* Mon Jun 14 2021 lingsheng <lingsheng@huawei.com> - 1.2.3-3
|
||||||
- Fix status failure after stopping service
|
- Fix status failure after stopping service
|
||||||
|
|
||||||
* Wed May 26 2021 wangchong <wangchong56@huawei.com> 1.2.3 - 2
|
* Wed May 26 2021 wangchong <wangchong56@huawei.com> - 1.2.3-2
|
||||||
- add requires
|
- add requires
|
||||||
|
|
||||||
* Thu Mar 18 2021 huanghaitao <huanghaitao8@huawei.com> 1.2.3 - 1
|
* Thu Mar 18 2021 huanghaitao <huanghaitao8@huawei.com> - 1.2.3-1
|
||||||
- Package init
|
- Package init
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user