Caused by: java.lang.RuntimeException: org.apache.flink.runtime.client.JobExecutionException: Could not set up JobManager
    at org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:36)
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
    ... 4 more
Caused by: org.apache.flink.runtime.client.JobExecutionException: Could not set up JobManager
    at org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:176)
    at org.apache.flink.runtime.dispatcher.Dispatcher$DefaultJobManagerRunnerFactory.createJobManagerRunner(Dispatcher.java:1058)
    at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:308)
    at org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:34)
    ... 7 more
Caused by: org.apache.flink.runtime.JobException: Creating the input splits caused an error: File /home/hadoop/data/1.log does not exist or the user running Flink ('hadoop') has insufficient permissions to access it.
    at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:276)
    at org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:830)
    at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:232)
    at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:100)
    at org.apache.flink.runtime.jobmaster.JobMaster.createExecutionGraph(JobMaster.java:1173)
    at org.apache.flink.runtime.jobmaster.JobMaster.createAndRestoreExecutionGraph(JobMaster.java:1153)
    at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:296)
    at org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:157)
    ... 10 more
Caused by: java.io.FileNotFoundException: File /home/hadoop/data/1.log does not exist or the user running Flink ('hadoop') has insufficient permissions to access it.
    at org.apache.flink.core.fs.local.LocalFileSystem.getFileStatus(LocalFileSystem.java:115)
    at org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:588)
    at org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:62)
    at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:262)
    ... 17 more

End of exception on server side>]
    at org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:380)
    at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$3(RestClient.java:364)
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
    ... 4 more

 

编写代码完成后,利用maven打包,将jar包上传到搭建好的集群中。到flink 的bin目录执行程序。

./bin/flink run -m hadoop01:8081 ./examples/batch/WordCount.jar --input /home/hadoop/data/1.log --output /home/hadoop/data/11.log
在上述命令中,-m是集群模式,-c是编写的WordCount的全类名,接着是jar包名字,input 和output是传入参数。出现下图表示运行成功,可以去WebUI查看具体日志。

standalone 模式,需要关闭hadoop,zookeeper
注意:传入的input参数必须在计算节点上有(计算节点为配置文件slave中写入的主机),否则会出现does not exist or the user running Flink ('hadoop') has insufficient permissions to access it. FileNotFoundException异常。

Logo

北京人形旗下天工造物具身智能开源社区,聚焦具身天工与慧思开物两大平台

更多推荐