idea removes spark logs

Spark will allow info, information by default, it has log4j built in

Therefore, we have 3 ways to solve it

1. Set the log level directly in the code, including adding in direct classes, and using inherited classes (not recommended)

2. Modify his global configuration (not recommended)

3. Add log4j.prperties

Note that there can be no other logs in the pom

Especially these two should be deleted, otherwise the log4j.proerpties you set will be invalid

    org.slf4jslf4j-simple1.7.25compileorg.slf4jslf4j-log4j12

Here we use the third one

Create a file like this with the same name

Add content to the file

log4j.rootCategory=ERROR, consolelog4j.appender.console=org.apache.log4j.ConsoleAppenderlog4j.appender.console.target=System.errlog4j.appender.console.layout=org.apache.log4j.PatternLayoutlog4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n# Set the default spark-shell log level to ERROR. When running the spark-shell, the # log level for this class is used to overwrite the root logger's log level, so that# the user can have different defaults for the shell and regular Spark apps.log4j.logger.org.apache.spark.repl.Main=ERROR# Settings to quiet third party logs that are too verboselog4j.logger.org.spark_project.jetty=ERRORlog4j.logger.org.spark_project.jetty.util.component.AbstractLifeCycle=ERRORlog4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=ERRORlog4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=ERRORlog4j.logger.org.apache.parquet=ERRORlog4j.logger.parquet=ERROR# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive supportlog4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATALlog4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR

Before the test

After the test