怎么在网上做装修网站,wordpress虚拟阅读量,网站建设教程照片,微网站方案报价如果您跟我一样#xff0c;目前正在使用Spring Cloud Config做为配置中心的话#xff0c;本篇将来要描述的问题#xff0c;强烈推荐了解和关注#xff01;因为这个问题目前存在于所有的版本中#xff0c;还没有完全修复。 问题现象
为了说明下面的内容#xff0c;我们可… 如果您跟我一样目前正在使用Spring Cloud Config做为配置中心的话本篇将来要描述的问题强烈推荐了解和关注因为这个问题目前存在于所有的版本中还没有完全修复。 问题现象
为了说明下面的内容我们可以先尝试重现一下问题在一个测试环境中将Spring Cloud Config的配置中心迁移到另外一个节点上即配置中心的IP地址发生了变化。在完成迁移之后我们会发现该环境下各个微服务应用的健康状态会变得时好时坏并且在日志中会出现类似下面的报错
2018-05-13 17:01:28,569 WARN [http-nio-9920-exec-1] org.springframework.cloud.config.client.ConfigServerHealthIndicator - Health check failedjava.lang.IllegalStateException: Could not locate PropertySource and the fail fast property is set, failing at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:132) at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator$$FastClassBySpringCGLIB$$fa44b2a.invoke(generated) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91) at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287) at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164) at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118) at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673) at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator$$EnhancerBySpringCGLIB$$3a43a1f4.locate(generated) at org.springframework.cloud.config.client.ConfigServerHealthIndicator.getPropertySource(ConfigServerHealthIndicator.java:54) at org.springframework.cloud.config.client.ConfigServerHealthIndicator.doHealthCheck(ConfigServerHealthIndicator.java:35) at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:43) at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) at org.springframework.boot.actuate.endpoint.HealthEndpoint.invoke(HealthEndpoint.java:85) at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.getCurrentHealth(HealthMvcEndpoint.java:177) at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.getHealth(HealthMvcEndpoint.java:166) at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(HealthMvcEndpoint.java:143) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.yonghui.feign.filter.RequestOriginFilter.doFilter(RequestOriginFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at com.yonghui.rpc.feature.web.boot.RpcHolder4BootFilter.doFilterInternal(RpcHolder4BootFilter.java:29) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at com.yonghui.rpc.feature.web.boot.FeatureSupport4BootFilter.doFilterInternal(FeatureSupport4BootFilter.java:24) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for http://192.168.5.103:9010/config-server/test: Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:674) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:621) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:539) at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:172) at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:93) ... 95 moreCaused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.init(HttpClient.java:242) at sun.net.www.http.HttpClient.New(HttpClient.java:339) at sun.net.www.http.HttpClient.New(HttpClient.java:357) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984) at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:78) at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:660) ... 99 more可以看到类似上面的健康检查失败错误但是并不是一直这样这个环境下的微服务会出现时好时坏的情况那么为什么会出现这种现象呢
原因分析
从错误日志中我们可以发现一个非常关键的信息I/O error on GET request for http://192.168.5.103:9010/config-server/test。
报错说明了微服务检查配置中心获取配置的连接是否畅通的时候出现了连接不上的情况但是这个链接信息其实并不是当前配置中心的地址而是我们迁移之前的配置中心的地址。
从健康检查的实现源码ConfigServerHealthIndicator中为入口去分析和调试我们可以解答上面现象的两个疑问 Override protected void doHealthCheck(Builder builder) throws Exception { PropertySource? propertySource getPropertySource(); builder.up(); if (propertySource instanceof CompositePropertySource) { ListString sources new ArrayList(); for (PropertySource? ps : ((CompositePropertySource) propertySource).getPropertySources()) { sources.add(ps.getName()); } builder.withDetail(propertySources, sources); } else if (propertySource!null) { builder.withDetail(propertySources, propertySource.toString()); } else { builder.unknown().withDetail(error, no property sources located); } }private PropertySource? getPropertySource() { long accessTime System.currentTimeMillis(); if (isCacheStale(accessTime)) { this.lastAccess accessTime; this.cached locator.locate(this.environment); } return this.cached;}为什么会健康检查访问的还是老的配置中心地址
真正导致健康检查失败的语句是getPropertySource中的 this.cached locator.locate(this.environment);而这里的具体实现在org.springframework.cloud.config.client.ConfigServicePropertySourceLocator类中具体实现如下
OverrideRetryable(interceptor configServerRetryInterceptor)public org.springframework.core.env.PropertySource? locate( org.springframework.core.env.Environment environment) { ConfigClientProperties properties this.defaultProperties.override(environment); CompositePropertySource composite new CompositePropertySource(configService); RestTemplate restTemplate this.restTemplate null ? getSecureRestTemplate(properties) : this.restTemplate; Exception error null; String errorBody null; logger.info(Fetching config from server at: properties.getRawUri()); ...}可以看到真正去访问的地址是直接从properties.getRawUri()获取的它已经是一个固化的值而不是通过服务发现机制来动态获取的。这就导致了当我们把配置中心做了迁移或者直接部署在容器中出现重启的时候IP发生变化而所有的微服务还以为访问的是原来的配置中心地址就会出现健康检查失败的问题导致服务不可用的现象。
为什么健康检查时好时坏
上面的问题会导致健康检查失败但是这个服务并不是一直都不好而是间断性的出现不健康。这主要还是健康检查时间中的机制导致这里可以具体看ConfigServerHealthIndicator的getPropertySource函数该方法执行的时候中间并不是每一次检查都会去访问配置中心执行locator.locate(this.environment)方法因此客户端的健康检查并不会每次都健康检查失败从而出现了微服务健康检查时好时坏的情况。
如何解决 该问题目前也在官方的issue中被提出还处于open状态具体可见https://github.com/spring-cloud/spring-cloud-config/issues/514 由于该问题目前并没有得到解决虽然提交了一个PR但是还有待完善以及提供一些测试原本想完全处理好之后再写一篇文章但是发现最近不少问过类似问题所以索性先写一篇文章提醒一下用户以及给一些相关的建议。
当前版本上不太容易通过扩展的方式去解决的这个问题所以大家可以变通的去避免这个问题
部署在虚拟机上而不是容器上避免IP的变动可以考虑关闭微服务队config客户端的健康检查增加参数health.config.enabledfalse但是这个操作有一个弊端虽然迁移不会引发服务时好时坏的问题了但是如果有动态配置刷新需求如果迁移了配置中心刷新配置操作也是会失败的。补充一个网友给的思路通过内部域名来绑定配置中心而不是ip地址
