好吧，我已经处理这个问题几天了，这让我发疯了。我需要使用带有事务的 Hive 数据库来执行“更新”和“删除”操作。

我已经以伪分布式模式在我的机器上安装了 Hadoop 和 Hive。我已关注此 tutorial用于安装。我使用的是 Java 1.8.0_31、Hadoop 2.6.0、Hive 1.0.0，我还更改了一些细节，但这些应该无关紧要。

现在，要启动我的环境(例如，重新启动后)，我运行以下命令:

start-dfs.sh
start-yarn.sh
java -jar /usr/local/derby/lib/derbyrun.jar server start &
hive

hive --hiveconf hive.root.logger=info,console 
    --hiveconf hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager 
    --hiveconf hive.compactor.initiator.on=true 
    --hiveconf hive.compactor.worker.threads=1 
    --hiveconf hive.txn.driver=jdbc:derby://localhost:1527/metastore_db;create=true

15/03/04 23:26:17 [main]: ERROR metastore.RetryingHMSHandler: 
    MetaException(message:Unable to select from transaction database, 
    java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does not exist.

hive --hiveconf hive.root.logger=info,console 
    --hiveconf hive.in.test=true 
    --hiveconf hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager 
    --hiveconf hive.compactor.initiator.on=true 
    --hiveconf hive.compactor.worker.threads=1 
    --hiveconf hive.txn.driver=jdbc:derby://localhost:1527/metastore_db;create=true;

ERROR metastore.RetryingHMSHandler: java.lang.NullPointerException
    at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:1146)

private void checkQFileTestHack() {
  boolean hackOn = HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEST) ||
    HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEZ_TEST);

  LOG.info("Before if");
  if (hackOn) {
      LOG.info("Hacking in canned values for transaction manager");
      // Set up the transaction/locking db in the derby metastore
      TxnDbUtil.setConfValues(conf);
      try {
          TxnDbUtil.prepDb();
      } catch (Exception e) {
          // We may have already created the tables and thus don't need to redo it.
          if (!e.getMessage().contains("already exists")) {
              throw new RuntimeException("Unable to set up transaction database for" +
                " testing: " + e.getMessage());
          }
      }
  }
}

mvn clean install -DskipTests -Phadoop-2,dist
rm /usr/local/hive/lib/hive-metastore-1.0.0.jar
cp packaging/target/apache-hive-1.0.0-bin/apache-hive-1.0.0-bin/lib/hive-metastore-1.0.0.jar /usr/local/hive/lib/

最佳答案

我关注了斯里尼瓦斯 advice并且错误消失了。我不再需要将“hive.in.test”属性设置为“true”，一切正常。

我仍然不知道为什么更改源不会影响 hive 程序的其余部分，但我有事务工作。

编辑:万一链接失效，这里有一个报价:

After extracting Hive version, you have to create Hive meta store

sudo apt-get install mysql-server
sudo service mysql start
sudo apt-get install libmysql-java
ln -s /usr/share/java/libmysql-java.jar /usr/lib/hive/lib/libmysql-java.jar
sudo chkconfig mysql on

mysql -u root -p
Enter password:
mysql> CREATE DATABASE metastore;
mysql> USE metastore;
mysql> SOURCE /usr/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-0.12.0.mysql.sql;

mysql> CREATE USER 'hive'@'metastorehost' IDENTIFIED BY 'mypassword';
...
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'metastorehost';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'metastorehost';
mysql> FLUSH PRIVILEGES;
mysql> quit;

Then in hive-site.xml, you need set the new parameters like

javax.jdo.option.ConnectionURL - jdbc:mysql://myhost/metastore
javax.jdo.option.ConnectionDriverName - com.mysql.jdbc.Driver
javax.jdo.option.ConnectionUserName - hive
javax.jdo.option.ConnectionPassword - mypassword
datanucleus.autoCreateSchema - false
datanucleus.fixedDatastore - true
datanucleus.autoStartMechanism - SchemaTable
hive.metastore.uris - thrift://<n.n.n.n>:9083

hive.support.concurrency – true  
hive.enforce.bucketing – true 
hive.exec.dynamic.partition.mode – nonstrict  
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager 
hive.compactor.initiator.on – true  
hive.compactor.worker.threads – 1

Then restart Hive-server and Metastore. Now create one normal table and one external table with orc format and load from normal to orc table. Now you can update and delete records.

关于java - Hive 事务正在崩溃，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/28867368/