常见Java内存数据库比较1. 为什么要内存数据库2. 有哪些常见的内存数据库3. 如何选择4. 示例代码

常见Java内存数据库比较1. 为什么要内存数据库2. 有哪些常见的内存数据库3. 如何选择4. 示例代码

1. 为什么要内存数据库 在常见的Java应用项目中,通常会有需求将数据保存到DB中。但实际环境中,受到服务器资源限制或者网络限制等因素,无法为项目提供DB资源。比如:

项目研发初期本地调试运行阶段,无法连接公司有网络访问控制的DB; 项目单元测试阶段,单测数据与正常测试数据污染隔离,清理测试库效率较低; 开发初学者用自己电脑运行程序,但又不想在本地安装MySQL导致电脑运行较慢; 云ECS服务器资源受限; 以上场景下需要DB又不能很好获取DB资源,所以会优先考虑内存型数据库。内存数据库一方面能满足应用对DB的需求,另一方面依赖资源(CPU和内存)简单,能及时打扫清理现场,认可度很高。

2. 有哪些常见的内存数据库 DB类型 优点 缺点 其它 sqlite 1. C语言开源,轻量级,通过jar方式引用;

2. 嵌入式模式运行,支持Java/go/python/php等多语言;

3. 嵌入式运行支持内存和磁盘模式,存储文件可以跨平台使用;

4. 支持实物隔离和索引;

1. 多线程并发读写能力弱;

2. Mysql兼容性较弱;

3. 无用户管理;

官网地址:SQLite Home Page H2 1. 纯Java编写,通过Jar方式引用;

2.支持client/server多线程模式;

3. 支持内存和磁盘存储数据;

4.支持索引和事务隔离,支持全文索引;

5. 相比于Sqlite对mysql语法支持更多;

6. 相同数据量级内存模式比较性能最好;

与SQLite类似 官网地址:H2 Database Engine derby 1. 开源Java语言编写,核心部分derby.jar只有2M;

2. 支持主从模式,支持授权用户;

1. derby 对很多 mysql 的关键字并不支持,同时 derby 不支持插入空值;

2. 只支持Java语言;

3. 不支持内存模式和全文本搜索;

官网地址:Apache Derby mariaDb4j 1. 号称能兼容MySQL的内存数据库;

2. 支持sql 索引;

1. 运行会依赖外部os的动态lib; github:MariaDB4j/mariaDB4j-pom-lite at master · vorburger/MariaDB4j · GitHub embeded mysql 1. 支持client/server模式;

2. 号称对mysql语法兼容性最强;

1. 只支持内存模式;

2. 5.8以后就被移除了,开源维护性较差;

3. 支持数据量级和性能受限,10万数据量级可能有问题;

github: GitHub - wix/wix-embedded-mysql: embedded mysql based on https://github.com/flapdoodle-oss/de.flapdoodle.embed.process 3. 如何选择 进考虑单机嵌入式情况,推荐使用sqlite,具体原因如上;

考虑事务支持以及内存情况下运行效率,选择H2;

如果考虑对MySQL语法支持,优先选择使用mariaDb4j,LZ亲测对sql语法兼容性强;

4. 示例代码 为了屏蔽底层DB引擎的差异性以及对查询的多线程使用,楼主选用了Druid作为数据库连接池,DAO层直接用Spring JDBC做封装。

公共依赖包:

com.alibaba

druid

1.1.2

org.springframework.boot

spring-boot-starter-jdbc

抽象公共的RbdUtil:

package com.book.xw.common.dal;

import com.alibaba.druid.pool.DruidDataSource;

import lombok.Data;

import org.springframework.jdbc.core.JdbcTemplate;

import java.sql.*;

import java.util.ArrayList;

import java.util.List;

import java.util.Map;

public abstract class RdbDaoUtil {

protected static String DB_LOCAL_PATH = "./mydb";

private JdbcTemplate jdbcTemplate;

private DruidDataSource dataSource;

protected String dbName;

private volatile Integer dataSourceStatus = 1;

public RdbDaoUtil(String dbName) {

this.dbName = dbName;

this.dataSource = buildRdbDataSource(dbName);

this.jdbcTemplate = new JdbcTemplate(dataSource);

}

protected abstract DruidDataSource buildRdbDataSource(String ... args);

// 保障不同数据库名字对应的实例只有一个

protected DruidDataSource buildDataSource(String dbUrl, String user, String pwd, String driverClass){

synchronized (dbName){

if(dataSourceStatus == 1){

DruidDataSource dataSource = new DruidDataSource();

dataSource.setUrl(dbUrl);

dataSource.setUsername(user);

dataSource.setPassword(pwd);

dataSource.setInitialSize(1);

dataSource.setMinIdle(1);

dataSource.setMaxWait(30000);

dataSource.setMinEvictableIdleTimeMillis(30000);

dataSource.setTestWhileIdle(true);

dataSource.setValidationQuery("select 1");

dataSource.setTestOnBorrow(true);

dataSource.setTestOnReturn(false);

dataSource.setLogAbandoned(true);

try {

dataSource.init();

}catch (Exception e){

throw new RuntimeException(e);

}

}else{

return this.dataSource;

}

}

return dataSource;

}

public void destroyDataSource(){

if(this.dataSource != null && !dataSource.isClosed()){

this.dataSource.close();

dataSourceStatus = 1;

}

}

public void createTable(String table, List columns ){

String sql = "";

jdbcTemplate.execute(sql);

}

public void deleteTable(String table){

String sql = "delete table if exists ` "+table+"` ;";

jdbcTemplate.update(sql);

}

public void executeSql(String sql){

jdbcTemplate.execute(sql);

}

public List> queryForMap(String sql){

return jdbcTemplate.queryForList(sql);

}

public MyData queryDbData(String sql, boolean needColName){

MyData data = new MyData();

Connection conn = null;

PreparedStatement stmt = null;

ResultSet rs = null;

try {

conn = this.dataSource.getConnection();

conn.setAutoCommit(false);

stmt = conn.prepareStatement(sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);

rs = stmt.executeQuery();

if(needColName){

getColNameAndType(rs, data);

}

int row = 0;

while (rs.next()){

getColData(rs, data);

row++;

}

data.setRows(row);

} catch (SQLException e) {

throw new RuntimeException(e);

} finally {

try {

if(rs != null){

rs.close();

}

if(stmt != null){

stmt.close();

}

if(conn != null){

conn.setAutoCommit(true);

conn.close();

}

} catch (SQLException e) {

e.printStackTrace();

}

}

return data;

}

private void getColData(ResultSet rs, MyData data){

int size = data.getColNames().size();

if(rs != null){

List list = new ArrayList<>(size);

try {

for(int i = 0; i< size; i++){

list.add(rs.getObject(i + 1));

}

} catch (SQLException e) {

throw new RuntimeException(e);

}

data.getColData().add(list);

}

}

private void getColNameAndType(ResultSet rs, MyData data){

List names = new ArrayList<>();

List types = new ArrayList<>();

if(rs != null){

try {

ResultSetMetaData rms = rs.getMetaData();

for(int i = 1; i<=rms.getColumnCount(); i++){

types.add(rms.getColumnType(i));

names.add(rms.getColumnLabel(i));

}

} catch (SQLException e) {

throw new RuntimeException(e);

}

}

data.setColNames(names);

data.setColTypes(types);

}

@Data

public class ColumnType{

// 字段名字

private String name;

// 字段类型

private String type;

// 约束条件,主键、普通索引

private String constraint;

}

@Data

public class MyData{

private List colNames;

private List colTypes;

private List> colData = new ArrayList<>();

private int rows ;

}

}

Embedded Msql需要引入的包为:

com.wix

wix-embedded-mysql

4.6.2

引擎代码:

import com.alibaba.druid.pool.DruidDataSource;

import com.book.xw.common.dal.RdbDaoUtil;

import com.wix.mysql.EmbeddedMysql;

import com.wix.mysql.config.Charset;

import com.wix.mysql.config.MysqldConfig;

import com.wix.mysql.distribution.Version;

import lombok.SneakyThrows;

import java.net.ServerSocket;

import java.util.TimeZone;

public class EmbeddedMysqlUtil extends RdbDaoUtil {

private EmbeddedMysql embeddedMysql;

public EmbeddedMysqlUtil(String dbName) {

super(dbName);

}

@Override

protected DruidDataSource buildRdbDataSource(String... args) {

MysqldConfig config = mysqldConfig();

embeddedMysql = EmbeddedMysql

.anEmbeddedMysql(mysqldConfig())

.addSchema(dbName)

.start();

String url = "jdbc:mysql://localhost:"+config.getPort()+"/"+dbName+"?useUnicode=true&characterEncoding=UTF-8";

String driverClass = "com.mysql.jdbc.driver";

return buildDataSource(url, mysqldConfig().getUsername(), mysqldConfig().getPassword(), driverClass);

}

@Override

public void destroyDataSource() {

super.destroyDataSource();

if(embeddedMysql != null){

embeddedMysql.stop();

}

}

private MysqldConfig mysqldConfig(){

return MysqldConfig.aMysqldConfig(Version.v5_7_latest)

.withCharset(Charset.UTF8)

.withPort(randomPort())

.withTimeZone(TimeZone.getDefault())

.withTempDir(DB_LOCAL_PATH)

.build();

}

@SneakyThrows

private int randomPort(){

try(ServerSocket serverSocket = new ServerSocket(0)){

return serverSocket.getLocalPort();

}

}

}

MariaDB4j引擎jar:

// 核心包

ch.vorburger.mariaDB4j

mariaDB4j-core

2.4.0

// linux os依赖包

ch.vorburger.mariaDB4j

mariaDB4j-db-linux64

10.2.11

// mac os 依赖包

ch.vorburger.mariaDB4j

mariaDB4j-db-mac64

10.2.11

引擎代码:

import ch.vorburger.mariadb4j.DB;

import ch.vorburger.mariadb4j.DBConfigurationBuilder;

import com.alibaba.druid.pool.DruidDataSource;

import com.book.xw.common.dal.RdbDaoUtil;

import lombok.SneakyThrows;

public class MariaDb4jUtil extends RdbDaoUtil {

private DB mariaDb;

public MariaDb4jUtil(String dbName) {

super(dbName);

}

@SneakyThrows

@Override

protected DruidDataSource buildRdbDataSource(String... args) {

buildDb();

mariaDb.createDB(dbName);

String url = "jdbc:mysql://localhost:"+mariaDb.getConfiguration().getPort()+"/"+dbName+"?useUnicode=true&characterEncoding=UTF-8";

String driverClass = "com.mysql.jdbc.driver";

return buildDataSource(url, "root", "", driverClass);

}

@SneakyThrows

private void buildDb(){

DBConfigurationBuilder builder = DBConfigurationBuilder.newBuilder();

// 0 -> auto detect free port

builder.setPort(0);

builder.setBaseDir(DB_LOCAL_PATH);

mariaDb = DB.newEmbeddedDB(builder.build());

mariaDb.start();

}

@SneakyThrows

@Override

public void destroyDataSource() {

super.destroyDataSource();

if(mariaDb != null){

mariaDb.stop();

}

}

}