How to Enable Data Integration Feature
- Data integration is an optional extension module of the HAP system, and users can decide whether to enable it based on actual needs.
- Before enabling, please ensure Docker version is 20.10.16 or above, and the server has at least 8GB of available physical memory.
- If the server deploying HAP has 32GB, please do not enable this module additionally, as it will worsen the memory shortage, affecting system availability. It is recommended to upgrade to 64GB before enabling.
For quick installation in standalone deployment mode, refer to the following: (cluster deployment mode is relatively complex and HAP team assistance is recommended)
-
Download the image (Offline package download)
docker pull nocoly/hap-flink:1.17.1.530
-
Execute the following command to check if there is an output containing the keyword
start minio
in one line of contentdocker logs $(docker ps | grep hap-sc | awk '{print $1}') | grep minio
- Output exists
- No output
-
Modify the docker-compose.yaml file (default path:
/data/hap/script/docker-compose.yaml
)-
Add flink service
flink:
image: nocoly/hap-flink:1.17.1.530
entrypoint: ["/bin/bash"]
command: ["/run.sh"]
environment:
ENV_FLINK_S3_ENDPOINT: "sc:9010"
volumes:
- ./volume/data/:/data/ -
Add the
ENV_FLINK_URL
environment variable to the app serviceapp:
environment:
ENV_FLINK_URL: "http://flink:8081"Example
version: '3'
services:
app:
image: nocoly/hap-community:6.3.3
environment: &app-environment
ENV_ADDRESS_MAIN: "https://hap.domain.com"
ENV_APP_VERSION: "6.3.3"
ENV_API_TOKEN: "******"
ENV_FLINK_URL: "http://flink:8081" #added content
ports:
- 8880:8880
volumes:
- ./volume/data/:/data/
- ../data:/data/hap/data
sc:
image: nocoly/hap-sc:3.1.0
environment:
<<: *app-environment
volumes:
- ./volume/data/:/data/
command:
image: nocoly/hap-command:node2011-python312
environment:
<<: *app-environment
doc:
image: nocoly/hap-doc:2.0.0
environment:
ENV_FILE_INNER_URI: "app:8880"
# following content is added
flink:
image: nocoly/hap-flink:1.17.1.530
entrypoint: ["/bin/bash"]
command: ["/run.sh"]
environment:
ENV_FLINK_S3_ENDPOINT: "sc:9010"
volumes:
- ./volume/data/:/data/
-
-
Restart HAP service
-
Modify the docker-compose.yaml file (default path:
/data/hap/script/docker-compose.yaml
)-
Add flink service
flink:
image: nocoly/hap-flink:1.17.1.530
entrypoint: ["/bin/bash"]
command: ["/run.sh"]
environment:
ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"
volumes:
- ./volume/data/:/data/ -
Add flink-minio service to independently deploy a minio service for flink to store related data
flink-minio:
container_name: flink-minio
image: nocoly/hap-minio:RELEASE.2025-04-22T22-12-26Z
environment:
MINIO_ROOT_USER: "mdstorage"
MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"
volumes:
- ./volume/data/:/data/
command: minio server /data/flink-minio --console-address ":9001" -
Add the
ENV_FLINK_URL
environment variable to the app serviceapp:
environment:
ENV_FLINK_URL: "http://flink:8081"Example
version: '3'
services:
app:
image: nocoly/hap-community:6.3.3
environment: &app-environment
ENV_ADDRESS_MAIN: "https://hap.domain.com"
ENV_APP_VERSION: "6.3.3"
ENV_API_TOKEN: "******"
ENV_FLINK_URL: "http://flink:8081" #added content
ports:
- 8880:8880
volumes:
- ./volume/data/:/data/
- ../data:/data/hap/data
sc:
image: nocoly/hap-sc:3.1.0
environment:
<<: *app-environment
volumes:
- ./volume/data/:/data/
command:
image: nocoly/hap-command:node2011-python312
environment:
<<: *app-environment
doc:
image: nocoly/hap-doc:2.0.0
environment:
ENV_FILE_INNER_URI: "app:8880"
# following content is added
flink:
image: nocoly/hap-flink:1.17.1.530
entrypoint: ["/bin/bash"]
command: ["/run.sh"]
environment:
ENV_FLINK_S3_ENDPOINT: "flink-minio:9000"
volumes:
- ./volume/data/:/data/
flink-minio:
container_name: flink-minio
image: nocoly/hap-minio:RELEASE.2025-04-22T22-12-26Z
environment:
MINIO_ROOT_USER: "mdstorage"
MINIO_ROOT_PASSWORD: "eBxExGQJNhGosgv5FQJiVNqH"
volumes:
- ./volume/data/:/data/
command: minio server /data/flink-minio --console-address ":9001"
-
-
Download minio image
docker pull nocoly/hap-minio:RELEASE.2025-04-22T22-12-26Z
-
If you cannot access the internet, please download the offline image file, upload it to the server, and import the image file. Offline image file download link:
https://pdpublic.nocoly.com/offline/hap-minio-linux-amd64-RELEASE.2025-04-22T22-12-26Z.tar.gz
-
-
Restart HAP service
-
During the restart, open a new terminal and execute the following commands to create buckets needed by flink
docker exec -it flink-minio bash
mc alias set myminio http://127.0.0.1:9000 mdstorage eBxExGQJNhGosgv5FQJiVNqH
mc mb myminio/mdoc