在公司测试环境由于内存不足导致kafka服务自动停止，再次启动时java服务日志存在报错：

2024-09-18 10:31:35.727  WARN 23061 --- [consumer-0-C-1] o.a.k.c.NetworkClient                    : [Consumer clientId=consumer-ori-datasource-group-132, groupId=ori-datasource-group] 3 partitions have leader brokers without a matching listener, including [LO__8693UNN5W5U1V-0, LO_8693UNN5W5U1V-2, LO_8693UNN5W5U1V-1]

且无法消费。
在重新部署kafka后，java服务报错:

2024-09-19 11:21:16.362  INFO 12450 --- [main] o.a.k.c.u.AppInfoParser                  : Kafka version: 2.6.1
2024-09-19 11:21:16.362  INFO 12450 --- [main] o.a.k.c.u.AppInfoParser                  : Kafka commitId: 6b2021cd52659cef
2024-09-19 11:21:16.362  INFO 12450 --- [main] o.a.k.c.u.AppInfoParser                  : Kafka startTimeMs: 1726716076361
2024-09-19 11:21:16.377  INFO 12450 --- [main] o.a.k.c.Metadata                         : [Consumer clientId=consumer-morphling-group-1, groupId=morphling-group] Cluster ID: iz6BuPwmSYq-5AVvZ6qg9A
2024-09-19 11:22:16.378  WARN 12450 --- [main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'morphlingSceneCreateEventListener': Invocation of init method failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
2024-09-19 11:22:16.384  INFO 12450 --- [main] o.s.s.q.SchedulerFactoryBean             : Shutting down Quartz Scheduler
2024-09-19 11:22:16.384  INFO 12450 --- [main] o.q.c.QuartzScheduler                    : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 shutting down.
2024-09-19 11:22:16.384  INFO 12450 --- [main] o.q.c.QuartzScheduler                    : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 paused.
2024-09-19 11:22:16.385  INFO 12450 --- [main] o.q.c.QuartzScheduler                    : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 shutdown complete.
2024-09-19 11:22:16.393  INFO 12450 --- [main] o.a.z.ZooKeeper                          : Session: 0x20b50402bf10e98 closed
2024-09-19 11:22:16.393  INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn                         : EventThread shut down for session: 0x20b50402bf10e98
2024-09-19 11:22:16.398  INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn                         : EventThread shut down for session: 0x10a719434c80e99
2024-09-19 11:22:16.398  INFO 12450 --- [main] o.a.z.ZooKeeper                          : Session: 0x10a719434c80e99 closed
2024-09-19 11:22:16.451  WARN 12450 --- [main] c._.i.l.f.LynxFlinkConfiguration         : Lynx is disabled, method call to YarnClient should not be made:
2024-09-19 11:22:16.451  WARN 12450 --- [main] c._.i.l.f.LynxFlinkConfiguration         : public void org.apache.hadoop.service.AbstractService.stop()([])
2024-09-19 11:22:16.477  INFO 12450 --- [Curator-Framework-0] o.a.c.f.i.CuratorFrameworkImpl           : backgroundOperationsLoop exiting
2024-09-19 11:22:16.486  INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn                         : EventThread shut down for session: 0x20b50402bf10e96
2024-09-19 11:22:16.486  INFO 12450 --- [main] o.a.z.ZooKeeper                          : Session: 0x20b50402bf10e96 closed
2024-09-19 11:22:16.488  INFO 12450 --- [main] o.a.k.c.p.KafkaProducer                  : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2024-09-19 11:22:16.497  INFO 12450 --- [main] c._.a.e.c.RestHighLevelClientFactory     : Shutting down elasticsearch client
2024-09-19 11:22:16.524  INFO 12450 --- [main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2024-09-19 11:22:16.527  INFO 12450 --- [main] c.z.h.HikariDataSource                   : HikariPool-1 - Shutdown initiated...
2024-09-19 11:22:16.549  INFO 12450 --- [main] c.z.h.HikariDataSource                   : HikariPool-1 - Shutdown completed.
2024-09-19 11:22:16.608  INFO 12450 --- [main] c.b.e.w.c.DefaultService                 : Stopping service [com.bes.Service]
2024-09-19 11:22:16.665  INFO 12450 --- [main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2024-09-19 11:22:16.676 ERROR 12450 --- [main] o.s.b.SpringApplication                  : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'morphlingSceneCreateEventListener': Invocation of init method failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:139) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
    at com._4paradigm.antifraud.gateway.GatewayApplication.main(GatewayApplication.java:28) [classes!/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_202]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_202]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

由于是测试环境，决定清理kafka及zookeeper数据。

清理数据

操作前建议停止调用kafka的服务，以免删除不掉。

检查topic
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181
删除topic
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "./bin/kafka-topics.sh --delete --zookeeper 18.1.1.1:2181,18.2.2.2:2181 --topic",$1}'
删除zookeeper数据
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "rmr  /brokers/topics/"$1}'
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "rmr  /admin/delete_topics/"$1}'
清理kafka数据
mv kafka-logs{,.bak}

清理数据及移除目录后，即可重启kafka，重新执行检查topic步骤，为空则表示完成。

参考文档：https://www.cnblogs.com/caoweixiong/p/11748180.html

END

本文标题：测试环境kafka服务异常重启后无法使用

本文作者：宇宙最帅的男人

本文链接：https://lolicp.com/linux/202419703.html

版权声明：转载或者引用本文内容请注明来源及原作者，本文著作权归 (lolicp.com) 所有。

除非另有说明，本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议。

最后修改：2024 年 09 月 19 日
© 允许规范转载
如果觉得我的文章对你有用，请随意赞赏