ElastiFlow:
ElastiFlow统一流收集器使用 IPFIX、Netflow 和 sFlow 接收、解码、转换、规范化、转换和丰富从网络设备和应用程序发送的网络流记录和遥测数据。生成的记录可以发送到各种Elasticsearch和服务,包括:
- 弹性搜索
- 弹性云
- 面向 Elasticsearch 的 Open Distro
- AWS 弹性搜索服务
Flow版本比较:
ElastiFlow安装:
注意:ElastiFlow需要和ElasticSearch(数据库)、kibana(图形化呈现)配合使用。
在宿主机CENTOS8上:
提前配置一下时间相关文件,后续同步到ELK容器中:
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
docker-compose.yml:
version: '2.2'
services:
# ElasticSearch容器相关定义内容,做收集流的数据库
elasticsearch:
# 镜像名称
image: elastic/elasticsearch:7.8.1
privileged: true
environment: # ES设置,这里并没有做集群
- discovery.type=single-node
- node.name=netdevops_es
- cluster.name=netdevops_es_cluster
- network.host=0.0.0.0
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms4g -Xmx4g" # 资源控制,这里给了4g内存,可以根据自己的设备性能进行调整
volumes:
- /usr/share/elasticsearch/data # 数据持久化
- /etc/timezone:/etc/timezone:ro # 调整容器内的时间
- /etc/localtime:/etc/localtime:ro
networks:
- elastiflow_net # 设置连接的网络
ports: # 端口映射
- "9200:9200"
- "9300:9300"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
restart: always
# 定义kibana容器,做最终的图形化界面呈现
kibana:
image: elastic/kibana:7.8.1
privileged: true
environment:
- SERVER_NAME=netdevops_kibana
- ELASTICSEARCH_URL=http://elasticsearch:9200
- PATH_DATA=/usr/share/kibana/data
- NODE_OPTIONS="--max_old_space_size=4096"
volumes:
- /usr/share/kibana/data # 数据持久化
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
networks:
- elastiflow_net
ports:
- "5601:5601"
ulimits:
memlock:
soft: -1
hard: -1
depends_on:
- "elasticsearch"
restart: always
# 通过elastiflow做netflow流量收集
elastiflow-logstash:
# 镜像名称
image: robcowart/elastiflow-logstash:4.0.1
container_name: elastiflow-logstash
volumes: # 同步时间
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
depends_on:
- elasticsearch
networks:
- elastiflow_net
ports:
- "2055:2055/udp"
- "6343:6343/udp"
- "4739:4739/udp"
environment:
# JVM Heap size - this MUST be at least 3GB (4GB preferred)
LS_JAVA_OPTS: '-Xms4g -Xmx4g' # 调整内存
# ElastiFlow global configuration
ELASTIFLOW_AGENT_ID: elastiflow
ELASTIFLOW_GEOIP_CACHE_SIZE: 16384
ELASTIFLOW_GEOIP_LOOKUP: 'true'
ELASTIFLOW_ASN_LOOKUP: 'true'
ELASTIFLOW_OUI_LOOKUP: 'false'
ELASTIFLOW_POPULATE_LOGS: 'true'
ELASTIFLOW_KEEP_ORIG_DATA: 'true'
ELASTIFLOW_DEFAULT_APPID_SRCTYPE: '__UNKNOWN'
# 定义数据的地址和端口
ELASTIFLOW_ES_HOST: 'elasticsearch:9200'
#ELASTIFLOW_ES_USER: 'elastic'
#ELASTIFLOW_ES_PASSWD: 'changeme'
#支持的三种流收集
ELASTIFLOW_NETFLOW_IPV4_PORT: 2055
ELASTIFLOW_NETFLOW_UDP_WORKERS: 2
ELASTIFLOW_NETFLOW_UDP_QUEUE_SIZE: 4096
ELASTIFLOW_NETFLOW_UDP_RCV_BUFF: 33554432
ELASTIFLOW_SFLOW_IPV4_PORT: 6343
ELASTIFLOW_SFLOW_UDP_WORKERS: 2
ELASTIFLOW_SFLOW_UDP_QUEUE_SIZE: 4096
ELASTIFLOW_SFLOW_UDP_RCV_BUFF: 33554432
ELASTIFLOW_IPFIX_UDP_IPV4_PORT: 4739
ELASTIFLOW_IPFIX_UDP_WORKERS: 2
ELASTIFLOW_IPFIX_UDP_QUEUE_SIZE: 4096
ELASTIFLOW_IPFIX_UDP_RCV_BUFF: 33554432
ulimits:
memlock:
soft: -1
hard: -1
restart: always
networks:
elastiflow_net:
driver: bridge
进入docker-compose.yml目录下使用如下命令部署容器(前提安装了docker和docker-compos):
docker-compose up -d
等待少许时间后,通过游览器,输入网站进入图形化界面,其中192.168.0.166为容器宿主机地址:
实验测试:
实验目的:
收集路由的Netflow信息,并用图形化呈现。
实验设备:
一台CSR1000v,完成初始化配置,保证能够和192.168.0.166(宿主机)通信。
实验步骤:
步骤一: 完成CSR1000v Netflow的配置
# 设置监控的流
flow record ElastiFlow-Record
match ipv4 source address
match ipv4 destination address
match ipv4 protocol
match transport destination-port
match transport source-port
match interface input
collect counter bytes
# 设置服务器地址和端口号,注意更换为自己的IP地址
flow exporter Netflow-Exporter
destination 192.168.0.166
transport udp 2055
template data timeout 30
# 将服务器和监控的流做一个绑定
flow monitor Monitor1
exporter Netflow-Exporter
record ElastiFlow-Record
!
# 监控具体的接口和方向
interface GigabitEthernet1
ip flow monitor Monitor1 input
ip flow monitor Monitor1 output
步骤二: 创建Kibana索引模式
图1:
图2:
图3:
图4:
步骤三: 导入官方dashboards模板(模板在文末参考资料的代码中,名称为elastiflow.kibana.7.8.x.ndjson)
导入完成后,可以在界面中看到多个dashboards信息。
步骤4: 查看自己dashboard信息
- 查看Overview信息:
结果如下:
查看Flows(client Server)bashboard:
流量详细信息:
参考资料:
视频教程:https://www.bilibili.com/video/BV1HK4y1p7a8
代码地址:https://gitee.com/qytang/ElasticFlow
官方文档:https://docs.elastiflow.com/docs/
暂无评论内容