How to Deploy Flink Service
- Flink service is an optional extension module of the HAP system. Users can decide whether to enable it based on their actual needs.
- Before enabling, please ensure Docker version is 20.10.16 or above and the server has at least 8GB of free physical memory.
- If the server deploying HAP has 32GB of memory, do not enable this module additionally, as it may worsen memory shortage and affect system availability. It's recommended to upgrade to 64GB before enabling.
For quick installation in standalone deployment mode, refer to the following instructions (Cluster deployment mode is relatively more complex; you may contact the HAP team for assistance):
-
Download the Flink image (Offline package download)
- Linux amd64
- Linux arm64
docker pull nocoly/flink:1.19.720docker pull nocoly/flink-arm64:- -
Run the following command to check whether the output contains a line with the keyword
start minio:docker logs $(docker ps | grep sc | awk '{print $1}') | grep minio- Output Found
- No Output
-
Modify the docker-compose.yaml file (Default path:
/data/hap/script/docker-compose.yaml)-
Add the Flink service
- Linux amd64
- Linux arm64
flink:image: nocoly/flink:1.19.720entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "sc:9010"volumes:- ./volume/data/:/data/flink:image: nocoly/flink-arm64:-entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "sc:9010"volumes:- ./volume/data/:/data/ -
Add the
ENV_FLINK_URLenvironment variable to the app serviceapp:environment:ENV_FLINK_URL: "http://flink:8081"Click to view the full modification example
- Linux amd64
- Linux arm64
version: '3'services:app:image: nocoly/hap:7.3.6environment: &app-environmentENV_ADDRESS_MAIN: "https://hap.domain.com"ENV_APP_VERSION: "7.3.6"ENV_API_TOKEN: "******"ENV_FLINK_URL: "http://flink:8081" # Added: Enable Flink serviceports:- 8880:8880volumes:- ./volume/data/:/data/- ../data:/data/hap/datasc:image: nocoly/sc:3.2.0environment:<<: *app-environmentvolumes:- ./volume/data/:/data/command:image: nocoly/command:node2011-python312doc:image: nocoly/doc:2.0.0# --- Newly added service ---flink:image: nocoly/flink:1.19.720entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "sc:9010"volumes:- ./volume/data/:/data/version: '3'services:app:image: nocoly/hap-arm64:-environment: &app-environmentENV_ADDRESS_MAIN: "https://hap.domain.com"ENV_APP_VERSION: "-"ENV_API_TOKEN: "******"ENV_FLINK_URL: "http://flink:8081" # Added: Enable Flink serviceports:- 8880:8880volumes:- ./volume/data/:/data/- ../data:/data/hap/datasc:image: nocoly/sc-arm64:-environment:<<: *app-environmentvolumes:- ./volume/data/:/data/command:image: nocoly/command-arm64:-doc:image: nocoly/doc-arm64:-# --- Newly added service ---flink:image: nocoly/flink-arm64:-entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "sc:9010"volumes:- ./volume/data/:/data/
-
-
Restart services by running the following in the installation manager's extracted directory:
bash service.sh restartall
-
Modify the docker-compose.yaml file (Default path:
/data/hap/script/docker-compose.yaml)-
Add the Flink service
- Linux amd64
- Linux arm64
flink:image: nocoly/flink:1.19.720entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"volumes:- ./volume/data/:/data/flink:image: nocoly/flink-arm64:-entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"volumes:- ./volume/data/:/data/ -
Add the flink-minio service to independently deploy a MinIO service for the Flink service to store relevant data:
- Linux amd64
- Linux arm64
flink-minio:container_name: flink-minioimage: nocoly/minio:RELEASE.2025-04-22T22-12-26Zenvironment:MINIO_ROOT_USER: "mdstorage"MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"volumes:- ./volume/data/:/data/command: minio server /data/flink-minio --console-address ":9001"flink-minio:container_name: flink-minioimage: nocoly/minio-arm64:RELEASE.2025-04-22T22-12-26Zenvironment:MINIO_ROOT_USER: "mdstorage"MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"volumes:- ./volume/data/:/data/command: minio server /data/flink-minio --console-address ":9001" -
Add the
ENV_FLINK_URLenvironment variable to the app serviceapp:environment:ENV_FLINK_URL: "http://flink:8081"Example
- Linux amd64
- Linux arm64
version: '3'services:app:image: nocoly/hap:7.3.6environment: &app-environmentENV_ADDRESS_MAIN: "https://hap.domain.com"ENV_APP_VERSION: "7.3.6"ENV_API_TOKEN: "******"ENV_FLINK_URL: "http://flink:8081" # New additionports:- 8880:8880volumes:- ./volume/data/:/data/- ../data:/data/hap/datasc:image: nocoly/sc:3.2.0environment:<<: *app-environmentvolumes:- ./volume/data/:/data/command:image: nocoly/command:node2011-python312doc:image: nocoly/doc:2.0.0# Newly added servicesflink:image: nocoly/flink:1.19.720entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"volumes:- ./volume/data/:/data/flink-minio:container_name: flink-minioimage: nocoly/minio:RELEASE.2025-04-22T22-12-26Zenvironment:MINIO_ROOT_USER: "mdstorage"MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"volumes:- ./volume/data/:/data/command: minio server /data/flink-minio --console-address ":9001"version: '3'services:app:image: nocoly/hap-arm64:-environment: &app-environmentENV_ADDRESS_MAIN: "https://hap.domain.com"ENV_APP_VERSION: "-"ENV_API_TOKEN: "******"ENV_FLINK_URL: "http://flink:8081" # New additionports:- 8880:8880volumes:- ./volume/data/:/data/- ../data:/data/hap/datasc:image: nocoly/sc-arm64:-environment:<<: *app-environmentvolumes:- ./volume/data/:/data/command:image: nocoly/command-arm64:-doc:image: nocoly/doc-arm64:-# Newly added servicesflink:image: nocoly/flink-arm64:-entrypoint: ["/bin/bash"]command: ["/run.sh"]environment:ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"volumes:- ./volume/data/:/data/flink-minio:container_name: flink-minioimage: nocoly/minio-arm64:RELEASE.2025-04-22T22-12-26Zenvironment:MINIO_ROOT_USER: "mdstorage"MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"volumes:- ./volume/data/:/data/command: minio server /data/flink-minio --console-address ":9001"
-
-
Download the MinIO image:
- Linux amd64
- Linux arm64
docker pull nocoly/minio:RELEASE.2025-04-22T22-12-26ZIf internet access is unavailable, first download the offline image file, upload it to the server, and then import the image file. Offline image file download link:
https://pdpublic.nocoly.com/offline/minio-linux-amd64-RELEASE.2025-04-22T22-12-26Z.tar.gzdocker pull nocoly/minio-arm64:RELEASE.2025-04-22T22-12-26ZIf internet access is unavailable, first download the offline image file, upload it to the server, and then import the image file. Offline image file download link:
https://pdpublic.nocoly.com/offline/minio-linux-arm64-RELEASE.2025-04-22T22-12-26Z.tar.gz -
Restart services by running the following in the installation manager's extracted directory:
bash service.sh restartall -
During the restart, open a new terminal and execute the following commands to create the bucket needed by Flink:
docker exec -it flink-minio bashmc alias set myminio http://127.0.0.1:9000 mdstorage eBxExGQJNhGosgv5FQJiVNqHmc mb myminio/mdoc