Guava Striped总结

Guava Striped总结

业务背景

最近在研究接口平台API的token的获取方法,类似微信token,有过期时间需要持久化并更新。

    Lock l = new ReentrantLock();
    l.lock();
    try {
    	if (isAccessTokenExpired()) {
           //update token
    	}
           // get token
    	} finally {
    	l.unlock();
    }

若全局只有一个token这样获取token,没有问题,但是若全局有tokenA,tokenB等等,则获取tokenA的时候tokenB需要等待。这个时候我们要细化锁的粒度。

Striped注释分析

google 出品必属精品。

首先阅读源码注释得到以下说明:

  • 细化锁的粒度并允许独立操作。
  • key-value数据结构,通过key获取对应的锁
    • 若key1.equals(key2)则striped.get(key1) == striped.get(key2)
    • 若!key1.equals(key2)则不保证striped.get(key1) != striped.get(key2),即key1和key2可能为同一把锁,暂且叫做锁冲突
    • stripes数量越少锁冲突概率越大
  • striped分为strong和weak
    • strong为提前初始化、强引用、不可回收
    • weak为延迟初始化、弱引用、可回收

Striped数据结构

striped的基础数据结构为key-value,以Striped为例。value为`ReentrantLock(false)`,key的数据结构分为strong和weak

  • strong的key的数据结构为Object[]
  • weak的key的数据结构根据striped的数量分为ConcurrentMap<Integer, Lock>AtomicReferenceArray<ArrayReference<? extends Lock>> 且都为弱引用

锁冲突问题

  private abstract static class PowerOfTwoStriped<L> extends Striped<L> {
    /** Capacity (power of two) minus one, for fast mod evaluation */
    final int mask;

    PowerOfTwoStriped(int stripes) {
      Preconditions.checkArgument(stripes > 0, "Stripes must be positive");
      this.mask = stripes > Ints.MAX_POWER_OF_TWO ? ALL_SET : ceilToPowerOfTwo(stripes) - 1;
    }

    @Override
    final int indexFor(Object key) {
      int hash = smear(key.hashCode());
      return hash & mask;
    }

    @Override
    public final L get(Object key) {
      return getAt(indexFor(key));
    }
  }

其中mask为大于参数stripes的最小2的N次方减一,即散列的地址为0到mask。当发生散列碰撞的时候,即发生锁冲突。

JDBC小结-execute

工作操作JDBC中发现一个错误,总结下。

  • 使用executeQuery() 执行update delete insert 的时候会报如下异常。
java.sql.SQLException: Can not issue data manipulation statements with executeQuery().
  • 使用executeUpdate() 执行select 的时候会报如下异常。
java.sql.SQLException: Can not issue executeUpdate() for SELECTs
  • 使用execute 执行select update delete insert 都可以正常执行。

使用JDBC操作的时候会使用StatementPreparedStatement ,我去看下源码是如何说明的。JDBC是连接多种数据库的统一标准,JDBC规定的标准由不同的数据库驱动负责具体实现。所以StatementPreparedStatement 都是借口,我们也只需要看下接口的注释就可以了。

PreparedStatement 继承于Statement

我先看看Statement 如何定义相关方法的 :

  • executeQuery 方法中异常说明:
     * @exception SQLException if a database access error occurs,
     * this method is called on a closed <code>Statement</code>, the given
     *            SQL statement produces anything other than a single
     *            <code>ResultSet</code> object, the method is called on a
     * <code>PreparedStatement</code> or <code>CallableStatement</code>

当使用executeQuery 方法只要产生一个不是ResultSet 的对象就会报SQLException 异常,即篇头错误。

  • executeUpdate方法中异常说明 ```java
    • @exception SQLException if a database access error occurs,
    • this method is called on a closed Statement, the given
    • SQL statement produces a ResultSet object, the method is called on a
    • PreparedStatement or CallableStatement ```

当使用executeUpdate 方法产生一个ResultSet 对象就会报SQLException 异常,即篇头错误。

是否产生ResultSet 对象是区分executeQueryexecuteUpdate 的关键。

为什么execute方法可以正常执行呢?

    /**
     * Executes the given SQL statement, which may return multiple results.
     * In some (uncommon) situations, a single SQL statement may return
     * multiple result sets and/or update counts.  Normally you can ignore
     * this unless you are (1) executing a stored procedure that you know may
     * return multiple results or (2) you are dynamically executing an
     * unknown SQL string.

execute 方法可以返回result sets and/or update counts 。所以execute 可以正常执行。

PreparedStatement 中关于各个方法的说明都与Statement 类似,小伙伴可以自行去看源码。

多读源码,好处多多 。

Enum Singleton!

Enum单例模式深入学习

直接上代码

public enum Test {	
	ONE;	
	public void doMethod(){
		System.out.println("123");
	}	
}

反编译后

public final class Test extends Enum
{

	public static final Test ONE;
	private static final Test ENUM$VALUES[];

	private Test(String s, int i)
	{
		super(s, i);
	}

	public void doMethod()
	{
		System.out.println("123");
	}

	public static Test[] values()
	{
		Test atest[];
		int i;
		Test atest1[];
		System.arraycopy(atest = ENUM$VALUES, 0, atest1 = new Test[i = atest.length], 0, i);
		return atest1;
	}

	public static Test valueOf(String s)
	{
		return (Test)Enum.valueOf(demo/Test, s);
	}

	static 
	{
		ONE = new Test("ONE", 0);
		ENUM$VALUES = (new Test[] {
			ONE
		});
	}
}

反编译后,发现Enum可以看做是一种单例模式的精简写法。

在查询ORACLE官网获取证明,地址Enum Types

  • The final clone method in Enum ensures that enum constants can never be cloned.
  • Reflective instantiation of enum types is prohibited.
  • Special treatment by the serialization mechanism ensures that duplicate instances are never created as a result of deserialization.

简单说就是

  • Enum不会被cloned
  • Enum不会被反射创建
  • Enum还支持序列化,且是同一个实例

继续看官网文档

In an enum declaration, a constructor declaration with no access modifiers is private.

Enum的构造函数一定是private

通过反编译Enum示例文件和查看Enum官方文档。总结如下:

  • Enum单例模式,是一种精简的单例写法。
  • 通过static块初始化,保证线程安全。
  • Enum单例模式,不会被cloned,不会被反射创建,支持序列化。

No operations allowed after statement closed !

记一次数据源问题排查

  • 错误日志如下:
[INFO] 2016-11-02 15:24:05 | Problem with checked-in Statement, discarding. |  com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:268)
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:408)
	at com.mysql.jdbc.Util.getInstance(Util.java:383)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1023)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:997)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:983)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:928)
	at com.mysql.jdbc.StatementImpl.checkClosed(StatementImpl.java:463)
	at com.mysql.jdbc.PreparedStatement.clearParameters(PreparedStatement.java:1113)
	at com.mchange.v2.c3p0.stmt.GooGooStatementCache.refreshStatement(GooGooStatementCache.java:614)
	at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:260)
	at com.mchange.v2.c3p0.impl.NewPooledConnection.checkinStatement(NewPooledConnection.java:301)
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.close(NewProxyPreparedStatement.java:1847)
	at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.ibatis.logging.jdbc.PreparedStatementLogger.invoke(PreparedStatementLogger.java:78)
	at com.sun.proxy.$Proxy64.close(Unknown Source)
	at org.apache.ibatis.executor.BaseExecutor.closeStatement(BaseExecutor.java:286)
	at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:65)
	at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325)
	at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:83)
	at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
	at com.github.pagehelper.SqlUtil._processPage(SqlUtil.java:401)
	at com.github.pagehelper.SqlUtil.processPage(SqlUtil.java:374)
	at com.github.pagehelper.PageHelper.intercept(PageHelper.java:254)
	at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
	at com.sun.proxy.$Proxy62.query(Unknown Source)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:77)
	at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
	at com.sun.proxy.$Proxy18.selectOne(Unknown Source)
	at org.mybatis.spring.SqlSessionTemplate.selectOne(SqlSessionTemplate.java:167)
	at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75)
	at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
	at com.sun.proxy.$Proxy42.selectTemplateById(Unknown Source)
	at com.lefu.letou.service.impl.CreateTemplateServiceImpl.selectTemplateById(CreateTemplateServiceImpl.java:54)
	at com.lefu.letou.controller.CreateTemplateController.selectTemplate(CreateTemplateController.java:105)
	at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:832)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:743)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:961)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:895)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:65)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
	at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
	at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
	at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
	at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:409)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1044)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[ERROR] 2016-11-02 15:24:05 | org.springframework.dao.RecoverableDataAccessException: 
### Error querying database.  Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 1,240,559 milliseconds ago.  The last packet sent successfully to the server was 1 milliseconds ago.

  • 关键日志信息:No operations allowed after statement closed

  • 问题分析:
    1 connection超时导致statement关闭。
    1.1 数据源连接的相关配置
    1.2 mysql连接配置
    2 mysql默认连接超时时间是8小时,但是问题在短时间内1小时就可以重现,所以我们先来排查缓存和数据源连接
  • 排查
    1 查询数据源(c3p0)的相关配置:
    maxIdleTime 连接的最大空闲时间1800,测试,无效。
    idleConnectionTestPeriod用来配置测试空闲连接的间隔时间1800,测试,无效。
    2 查询数据库(mysql)相关配置:
    SHOW GLOBAL VARIABLES LIKE 'wait_timeout'; 发现了问题:
    wait_timeout | 600
    说好的mysql默认连接28800秒8小时呢?思维惯性害死人啊。
    因为c3p0的maxIdleTimeidleConnectionTestPeriod的时间都大于600实际上是没有生效的。 将这两个参数设置都小于600秒问题解决。
  • 总结
    数据连接池的设置都依附于数据库本身,排查问题应该先查看数据的配置。

spring-boot-starter-parent配置问题

在使用springboot的maven构建工程的时候,有两种方法,官方文档中
官网Maven配置
第一种为

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.4.1.RELEASE</version>
</parent>

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
    </plugins>
</build>

这种方法会自动配置相关属性spring-boot-maven-plugin不需要配置具体属性就可以编译出可执行的jar/war,但是需要parent标签,详情见官网Maven配置, 第二种为

<dependencyManagement>
     <dependencies>
        <dependency>
            <!-- Import dependency management from Spring Boot -->
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-dependencies</artifactId>
            <version>1.4.1.RELEASE</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

第二种方法,配置后再IDE(eclipse)中可以执行,但是编译成jar文件后java -jar会报错没有mainclass,需要对spring-boot-maven-plugin进行具体配置

   <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <maimClass>${start-class}</maimClass>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

=spring-boot-maven-plugin 官方API

springboot issue!

The space in the configuration file will result in different running results(eclipse and jar).

I find a case: version.1.4.1.RELEASE.version

in the application.properties

logging.config=classpath:log4j2.xml 

There is a space behind the ‘log4j2.xml’,the configuration can work in the eclipse ,but run the configuration as a jar (java -jar) it do not work!

Logging system failed to initialize using configuration from 'classpath:log4j2.xml 'java.io.FileNotFoundException: class path resource [log4j2.xml ] cannot be resolved to URL because it does not exist at org.springframework.util.ResourceUtils.getURL(ResourceUtils.java:138)

delete the space behind the ‘log4j2.xml’,it works as a jar.

is ok?

Submitted to github issue

Hello World!

Hello World 相传古时候有个退休的程序员,在家闲来无事,决定修习书法之道。第一日,备好笔墨纸砚,便挥毫写下一行大字:“hello, world”。 呵呵,上面是一个笑话。 hello world的起源要追溯到1972年,贝尔实验室著名研究员Brian Kernighan在撰写“B语言教程与指导(Tutorial Introduction to the Language B)”时初次使用(程序),这是目前已知最早的在计算机著作中将hello和world一起使用的记录。之后,在1978年,他在他和Dennis Ritchie合作撰写的C语言圣经“The C Programming Language”中,延用了“hello,world”句式,作为开篇第一个程序。在这个程序里,输出的”hello,world” 全部是小写,没有感叹号,逗号后有一空格 。虽然之后几乎没能流传下来这个最初的格式,但从此用hello world向世界打招呼成为惯例

happens-before!

《JSR-133:Java Memory Model and Thread Specification》定义happens-before规则

  1. 程序顺序规则:一个线程中的每个操作,happens-before与该线程中的任意后续操作。
  2. 监视器锁规则:对一个锁的解锁,happens-before于随后对这个锁的加锁。
  3. volatile变量规则:对一个volatile域的写,happens-before于任意后续对这个volatile域的读。
  4. 传递性规则:如果Ahappens-beforeB,且Bhappens-beforeC,那么Ahappens-beforeC。
  5. start()规则:如果线程A执行操作ThreadB.start()(启动线程B),那么线程A的ThreadB.start()操作happens-before于线程B中的任意操作。
  6. join()规则:如果线程A执行操作ThreadB.join()并层高返回,那么线程B中的任意操作happens-before于线程A从ThreadB.join()操作成功返回。