当前位置：网站首页>Azkaban recompile, solve: could not connect to SMTP host: SMTP 163.com, port: 465 [January 10, 2022]

Azkaban recompile, solve: could not connect to SMTP host: SMTP 163.com, port: 465 [January 10, 2022]

2022-04-23 20:12:00 【My brother is not strong enough to fight】

Azkaban Configuration file modification 465 The port keeps reporting errors , as follows ：

javax.mail.MessagingException: Could not connect to SMTP host: smtp.163.com, port: 465, response: -1
        at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1960)
        at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:642)
        at javax.mail.Service.connect(Service.java:295)
        at azkaban.utils.JavaxMailSender.connect(JavaxMailSender.java:34)
        at azkaban.utils.EmailMessage.connectToSMTPServer(EmailMessage.java:220)
        at azkaban.utils.EmailMessage.retryConnectToSMTPServer(EmailMessage.java:230)
        at azkaban.utils.EmailMessage.sendEmail(EmailMessage.java:213)
        at azkaban.utils.Emailer.sendEmail(Emailer.java:220)
        at azkaban.utils.Emailer.alertOnSuccess(Emailer.java:153)
        at azkaban.executor.ExecutionControllerUtils.alertUserOnFlowFinished(ExecutionControllerUtils.java:140)
        at azkaban.executor.ExecutionFinalizer.finalizeFlow(ExecutionFinalizer.java:97)
        at azkaban.executor.RunningExecutionsUpdater.updateExecutions(RunningExecutionsUpdater.java:130)
        at azkaban.executor.RunningExecutionsUpdaterThread.run(RunningExecutionsUpdaterThread.java:54)
2022/01/08 15:09:01.807 +0800 ERROR [EmailMessage] [Azkaban] Connecting to SMTP server failed, attempt: 4

The most useful feeling

During this period of time, an offline warehouse has been built , The scheduler uses Azkaban, In the last part , Yes Azkaban The problem that the alarm email can't be sent . To solve this problem , It took half a day ：

1） Alibaba cloud 22 Port is not open ：

Because of my hadoop The cluster environment is built on Alibaba cloud , After checking the data, it is learned that Alibaba cloud has put 22 Port disabled . The official alternative is to use 465 port

2） Directly modifying Azkaban The configuration file is 465 port ：

I look through Azkaban-web Source code , It does support the modification of port configuration file , Then I'll add the port and restart , Reexecute the task , Who knows, it's still wrong .

mail.port=465

3） adopt javaMail Eliminate the problem of mailbox

Start troubleshooting , Began to doubt my 163 There is a problem with the mailbox , I wrote a note casually javaMail, After testing, the mailbox is OK

package com.sonkwo.javamail;

import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import java.io.FileOutputStream;
import java.io.OutputStream;
import java.util.Date;
import java.util.Properties;

public class TestJavaMail {
    //  From   mailbox   and   password （ Replace with your email and password ）
    // PS:  Some mailbox servers increase the security of the password of the mailbox itself , to  SMTP  The client has set a separate password （ Some e-mails are called “ Authorization code ”）,
    //      For mailbox with independent password enabled ,  The mailbox password here must use this independent password （ Authorization code ）.
    public static String myEmailAccount = "[email protected]";
    public static String myEmailPassword = "xxxxxxx";

    //  Sender's email address  SMTP  Server address ,  Must be accurate ,  Different email server addresses are different ,  commonly ( Just general ,  Not absolutely ) The format is : smtp.xxx.com
    //  NetEase 163 Mailbox  SMTP  The server address is : smtp.163.com
    public static String myEmailSMTPHost = "smtp.163.com";

    //  Recipient email （ Replace with a valid email that you know ）
    public static String receiveMailAccount = "[email protected]";

    public static void main(String[] args) throws Exception {
        // 1.  Create parameter configuration ,  Parameter configuration for connecting to mail server 
        Properties props = new Properties();                    //  Parameter configuration 
        props.setProperty("mail.transport.protocol", "smtp");   //  Protocol used （JavaMail Specification requirements ）
        props.setProperty("mail.smtp.host", myEmailSMTPHost);   //  Of the sender's mailbox  SMTP  Server address 
        props.setProperty("mail.smtp.auth", "true");            //  Need to request authentication 

        // PS:  Some mailbox servers require  SMTP  Connection needs to use  SSL  Safety certification  ( To improve safety ,  Mailbox support SSL Connect ,  You can open it yourself ),
        //      If you can't connect to the mail server ,  Check the console print carefully  log,  If there is something similar  “ The connection fails ,  requirement  SSL  A secure connection ”  Such mistakes ,
        //      Open up below  /* ... */  Comment code between ,  Turn on  SSL  A secure connection .
        /*
        // SMTP  The port of the server  ( Not  SSL  The port of the connection is usually... By default  25,  Can not add ,  If it's on  SSL  Connect ,
        //                   It needs to be changed to  SMTP  The port of the server ,  For details, please refer to the help of the corresponding mailbox service ,
        //                  QQ Mailbox SMTP(SLL) Port is 465 or 587,  Check other mailboxes by yourself )
        final String smtpPort = "465";
        props.setProperty("mail.smtp.port", smtpPort);
        props.setProperty("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
        props.setProperty("mail.smtp.socketFactory.fallback", "false");
        props.setProperty("mail.smtp.socketFactory.port", smtpPort);
        */

        // 2.  Create session objects based on configuration ,  For interacting with the mail server 
        Session session = Session.getDefaultInstance(props);
        session.setDebug(true);                                 //  Set to debug Pattern ,  You can see the detailed sending  log

        // 3.  Create an email 
        MimeMessage message = createMimeMessage(session, myEmailAccount, receiveMailAccount);

        // 4.  according to  Session  Get mail transfer object 
        Transport transport = session.getTransport();

        // 5.  Use   Email account   and   password   Connect to the mail server ,  The mailbox certified here must match the  message  The sender's mailbox in is consistent ,  Otherwise, the report will be wrong 
        //
        //    PS_01:  The key to success or failure is ,  If the connection to the server fails ,  Will output the corresponding failure reason in the console  log,
        //            Take a closer look at the cause of the failure ,  Some mailbox servers will return error codes or view links of error types ,  According to the given error 
        //            Type to the help website of the corresponding mail server to see the specific failure reason .
        //
        //    PS_02:  The reasons for connection failure are usually as follows ,  Check the code carefully :
        //           (1)  The mailbox is not open  SMTP  service ;
        //           (2)  Wrong email password ,  For example, some e-mail boxes have their own passwords turned on ;
        //           (3)  Mailbox server is required to use  SSL  A secure connection ;
        //           (4)  Request too often or for other reasons ,  Rejected by mail server ;
        //           (5)  If all of the above are true ,  Go to the mail server website for help .
        //
        //    PS_03:  If you look carefully, log,  Seriously look at log,  understand log,  The reasons for the mistakes are log It has been stated that .
        transport.connect(myEmailAccount, myEmailPassword);

        // 6.  Send E-mail ,  Send to all receiving addresses , message.getAllRecipients()  Get all the recipients that were added when the mail object was created ,  Cc people ,  Send off the people 
        transport.sendMessage(message, message.getAllRecipients());

        // 7.  Close the connection 
        transport.close();
    }

    /**
     *  Create a simple message that contains only text 
     *
     * @param session  Session with server interaction 
     * @param sendMail  Sender email 
     * @param receiveMail  Recipient email 
     * @return
     * @throws Exception
     */
    public static MimeMessage createMimeMessage(Session session, String sendMail, String receiveMail) throws Exception {
        // 1.  Create an email 
        MimeMessage message = new MimeMessage(session);

        // 2. From:  Sender 
        message.setFrom(new InternetAddress(sendMail, " A treasure net ", "UTF-8"));

        // 3. To:  The recipient （ Multiple recipients can be added 、 CC 、 Send off ）
        message.setRecipient(MimeMessage.RecipientType.TO, new InternetAddress(receiveMail, "XX user ", "UTF-8"));

        // 4. Subject:  Email subject 
        message.setSubject(" A great discount ", "UTF-8");

        // 5. Content:  Message body （ have access to html label ）
        message.setContent("XX Hello, user ,  Today the whole show 5 fold ,  Hurry up and buy ,  Miss today and wait another year ...", "text/html;charset=UTF-8");

        // 6.  Set delivery time 
        message.setSentDate(new Date());

        // 7.  Save settings 
        message.saveChanges();

        return message;
    }
}

4） O & M excludes Alibaba cloud 465 Port problem

I feed back to the students of operation and maintenance and say my 163 No problem with the mailbox , Then he wrote a program test 465 The port can indeed send mail . This concludes that Azkaban Problems with components , Revised 465 Port does not take effect .

5） Make up your mind , compile Azkaban

The thought of recompiling Software , I feel like I have to download a lot of packages , You have to make a lot of mistakes , Have a great sense of frustration . But think about it , There is no other way now , Try bai .

6） Compile the complete , I feel very rewarding

little does one think Azkaban Compile time is so simple , It's time to get off work , Very excited. , Let's start .

1. Basic environment preparation

1） Download the source code

because github Download is slow , So I found a copy of the source code download on the code cloud .

[[email protected] software]$ git clone https://gitee.com/leechi78/azkaban.git

2） View this version azkaban Corresponding gradle Version is gradle-5.0-all.zip

[[email protected] software]$ cd azkaban

[[email protected] azkaban]$ cat ./gradle/wrapper/gradle-wrapper.properties

# Licensed under the Apache License, Version 2.0 (the "License"); you may not

# use this file except in compliance with the License. You may obtain a copy of

# the License at

# http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT

# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the

# License for the specific language governing permissions and limitations under

# the License.

distributionBase=GRADLE_USER_HOME

distributionPath=wrapper/dists

# To change gradle version, use (rather than changing the line below):

# ./gradlew wrapper --gradle-version=<version>

# e.g.

# ./gradlew wrapper --gradle-version=5.0

distributionUrl=https\://services.gradle.org/distributions/gradle-5.0-all.zip

zipStoreBase=GRADLE_USER_HOME

zipStorePath=wrapper/dists

3） download gradle

wget https://services.gradle.org/distributions/gradle-5.0-all.zip

[email protected] wrapper]$ cd /opt/software/azkaban/gradle/wrapper

[email protected] wrapper]$ cp /opt/software/gradle-5.0-all.zip ./

4） Extract to the specified directory

[[email protected] software]$ unzip -o gradle-5.0-all.zip -d /opt/module/

5） Configure environment variables

[[email protected] ~]$ sudo vim /etc/profile.d/my_env.sh

add to

# GRADLE_USER_HOME

export GRADLE_USER_HOME=/opt/module/gradle-5.0

export PATH=$PATH:$GRADLE_USER_HOME/bin

Make configuration effective

[[email protected] ~]$ source /etc/profile

2. Modify the configuration file

1） Modify the configuration file gradle-wrapper.properties Medium distributionUrl attribute , Indicate the use of local gradle

[[email protected] azkaban]$

vim ./gradle/wrapper/gradle-wrapper.properties

distributionUrl=distributionUrl=gradle-5.0-all.zip

zipStoreBase=GRADLE_USER_HOME

2） If not installed Git, Install ; otherwise , Installation free .

[[email protected] software]$ yum install git

3） stay azkaban-common/src/main/java/azkaban/utils/EmailMessage.java Medium sendEmail() In the method , Add the following code ：

[[email protected] azkaban]$ vim azkaban-common/src/main/java/azkaban/utils/EmailMessage.java

Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

final String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";

props.setProperty("mail.smtp.socketFactory.class", SSL_FACTORY);

props.setProperty("mail.smtp.socketFactory.fallback", "false");

props.setProperty("mail.smtp.socketFactory.port", "465");

[[email protected] azkaban]$ vim azkaban-common/src/main/java/azkaban/jobExecutor/ProcessJob.java

3. compile

1） Start compilation

[[email protected] azkaban]$ ./gradlew build installDist -x test

2） Compiler error

[[email protected] azkaban]$ ./gradlew build installDist -x test

Downloading file:/opt/software/azkaban/gradle/wrapper/gradle-5.0-all.zip

...........................................................................................................................

Unzipping /opt/module/gradle-5.0/wrapper/dists/gradle-5.0-all/8subxm1j4sfxdrf9xyahjxjyp/gradle-5.0-all.zip to /opt/module/gradle-5.0/wrapper/dists/gradle-5.0-all/8subxm1j4sfxdrf9xyahjxjyp

Set executable permissions for: /opt/module/gradle-5.0/wrapper/dists/gradle-5.0-all/8subxm1j4sfxdrf9xyahjxjyp/gradle-5.0/bin/gradle

Welcome to Gradle 5.0!

Here are the highlights of this release:

- Kotlin DSL 1.0

- Task timeouts

- Dependency alignment aka BOM support

- Interactive `gradle init`

For more details see Gradle 5.0 Release Notes

Starting a Gradle Daemon (subsequent builds will be faster)

Configuration on demand is an incubating feature.

> Task :az-exec-util:linkMainExecutable FAILED

> Task :az-core:compileJava