当前位置:网站首页>Nifi fast installation and file synchronization

Nifi fast installation and file synchronization

2022-04-23 16:43:00 A hundred nights

Nifi In short, it is a data pull 、 A system for data processing and data distribution .
Nifi Running requires JVM, need jdk,nifi 1.x The above needs jdk8 above ,0.x need jdk7 above .
This article is in Centos 7 Install in nifi-1.8.0-bin.tar.gz
Download address :nifi.apache.org/download (xx.tar.gz yes Linux edition ,xx.zip yes Windows edition )
User manual :http://nifi.apache.org/docs.html

install

1. Download installation package

wget -b http://mirror.bit.edu.cn/apache/nifi/1.8.0/nifi-1.8.0-bin.tar.gz

2. Unzip the installation package

tar -zxf nifi-1.8.0-bin.tar.gz

3. modify nifi/conf Under the table of contents nifi.properties file
Modify host number and port number
nifi.web.http.host= Host number
nifi.web.http.port= Service port number [ Customize ]

# web properties #
nifi.web.war.directory=./lib
nifi.web.http.host=192.168.197.106   //  Change to your own host ip
nifi.web.http.port=8182			     //  Default 8080, Prevent port occupation , It can be modified to other ports 
nifi.web.http.network.interface.default=
nifi.web.https.host=
nifi.web.https.port=
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=

4. Service startup
Get into nifi/bin Catalog , function nifi.sh file , The front and rear operation commands are as follows

./nifi.sh run 	// The front desk operation 
./nifi.sh start // Background operation 

The following prompt appears: the service is started successfully ,


Java home: /opt/soft/jdk180
NiFi home: /opt/soft/nifi-1.8.0

Bootstrap Config File: /opt/soft/nifi-1.8.0/conf/bootstrap.conf

see nifi Start state

./nifi.sh stauts
-----------------------------------------------------------------------------------------------------------

Java home: /opt/soft/jdk180
NiFi home: /opt/soft/nifi-1.8.0

Bootstrap Config File: /opt/soft/nifi-1.8.0/conf/bootstrap.conf

2021-05-01 17:24:49,589 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is currently running, listening to Bootstrap on port 43069, PID=2099

5. Service closing
If it's a foreground operation , direct Ctrl+C Exit Service .
If it's running in the background , Access to nifi/bin Catalog , Execute the following command

./nifi.sh stop

6.web UI
Service startup 1-5 Minutes later , Can open web see ,http:192.168.197.106:8182/nifi, As shown in the figure .
 Insert picture description here

Use

File synchronization

Realization : take /opt/act.log Synchronize file contents to /opt/data Empty directory , And keep the source file .
Processor required GetFile ( Read and delete files ) + PutFile( Save the file )

Drag and drop a... In the toolbar Processor Go to the canvas , Input ’local’, choice ’GetFile’ processor , Click on ADD Add to canvas , The same operation creates a ’PutFile’ processor .
 Insert picture description here

To configure GetFile processor

1. Select processor , Right click selection “configure”,“settings” The admission “success”, Automatically terminate the relationship .
 Insert picture description here
2.“properties” Attribute configuration .

  • Input Directory: Enter the source directory
  • File Filter: File filter
  • Keep Source File: Do you want to keep the source file , Default false
  • Recurse Subdirectories: Whether to pull the files in the subdirectory , Default true

After configuration , Click on ‘apply’ Save application .
 Insert picture description here

To configure PutFile processor

1. Select processor , Right click selection “configure”, stay “settings” The admission “failure” and “success”.
 Insert picture description here

2.“properties” Attribute configuration

  • Directory: Target directory
  • Conflict Resolution Strategy: Specify the operation after a file with the same name exists . Optional value :replace Replace existing files ,fail Execution failure , Don't generate files ,ignore Ignore .
  • Create Missing Directories: Specify whether the target directory does not exist or not to create a directory . Optional value :true newly build ,false No new .
    After configuration , Click on ‘apply’ Save application .
     Insert picture description here

Connecting the processor

1. Move the mouse to ’GetFile’ On , The following arrow appears , Click and drag to ’PutFile’ On .
 Insert picture description here
2. To configure “Settings”, Set up name:demo1, Increase priority ,‘FirstInFirstOutPrioritizer’. Click on ’add’, Create connection .
 Insert picture description here
3. Right click the canvas and select ’Start‘, Visible data flow .GetFile Read file output 7.36KB,PutFile Write files to the target directory ,7.36KB, Also visible in the target directory act.log file .
 Insert picture description here

版权声明
本文为[A hundred nights]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204231401521069.html