参考flink-operations-playground一步步操作

基本概念

A Savepoint is a consistent snapshot of the complete application state at a well-defined, globally consistent point in time (similar to a checkpoint)

API设计

构造和启动

git clone --branch release-1.10 https://github.com/apache/flink-playgrounds.git
cd flink-playgrounds/operations-playground
docker-compose build
docker-compose up -d

查看容器

liyongquan@liyongquan-server2:/3rd/flink-playgrounds/operations-playground$ docker-compose ps
                    Name                                  Command               State                   Ports                
-----------------------------------------------------------------------------------------------------------------------------
operations-playground_clickevent-generator_1   /docker-entrypoint.sh java ...   Up       6123/tcp, 8081/tcp                  
operations-playground_client_1                 /docker-entrypoint.sh flin ...   Exit 0                                       
operations-playground_jobmanager_1             /docker-entrypoint.sh jobm ...   Up       6123/tcp, 0.0.0.0:8081->8081/tcp    
operations-playground_kafka_1                  start-kafka.sh                   Up       0.0.0.0:9094->9094/tcp              
operations-playground_taskmanager_1            /docker-entrypoint.sh task ...   Up       6123/tcp, 8081/tcp                  
operations-playground_zookeeper_1              /bin/sh -c /usr/sbin/sshd  ...   Up       2181/tcp, 22/tcp, 2888/tcp, 3888/tcp

停止

docker-compose down -v

日志查看

jobmanager

docker-compose logs -f jobmanager

taskmanager

docker-compose logs -f taskmanager

调用flink-client

docker-compose run --no-deps client flink --help

REST API

查看job

curl localhost:8081/jobs

KAFKA TOPIC

输入

docker-compose exec kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 --topic input

输出

//output topic (24 records/min)
docker-compose exec kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 --topic output

List Running Jobs

liyongquan@liyongquan-server2:/3rd/flink-playgrounds/operations-playground$ docker-compose run --no-deps client flink list
Waiting for response...
------------------ Running/Restarting Jobs -------------------
28.05.2020 07:25:58 : 8e7edff5b8ff8375e6aacc73ae598853 : Click Event Count (RUNNING)
--------------------------------------------------------------
No scheduled jobs.

失败和故障恢复

观察输出结果

docker-compose exec kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 --topic output

kill taskmanager

docker-compose kill taskmanager

taskmanager 恢复

docker-compose up -d taskmanager