上海做设计公司网站,做网站的哪家好,微信开发文档小程序,网站开发的图标1.Hive简介 起源自facebook由Jeff Hammerbacher领导的团队 构建在Hadoop上的数据仓库框架 设计目的是让SQL技能良好#xff0c;但Java技能较弱的分析师可以查询海量数据 2008年facebook把hive项目贡献给Apache Hive的组件与体系架构 用户接口#xff1a;shell, thrift, web等…1.Hive简介 起源自facebook由Jeff Hammerbacher领导的团队 构建在Hadoop上的数据仓库框架 设计目的是让SQL技能良好但Java技能较弱的分析师可以查询海量数据 2008年facebook把hive项目贡献给Apache Hive的组件与体系架构 用户接口shell, thrift, web等 Th1.Hive简介 起源自facebook由Jeff Hammerbacher领导的团队 构建在Hadoop上的数据仓库框架 设计目的是让SQL技能良好但Java技能较弱的分析师可以查询海量数据 2008年facebook把hive项目贡献给ApacheHive的组件与体系架构 用户接口shell, thrift, web等 Thrift服务器 元数据库“Derby, Mysql等 解析器 HadoopHive安装模式 内嵌模式元数据保持在内嵌的Derby模式只允许一个会话连接(默认) 本地独立模式在本地安装Mysql把元数据放到Mysql内 远程模式元数据放置在远程的Mysql数据库2.配置Mysql1:copy mysql-connector-java-5.1.6-bin.jar到$HIVE_HOME/lib[jifengjifeng02 hadoop]$ ls7287OS_Code hadoop-1.2.1.tar.gz hive-0.12.0-bin tmphadoop-1.2.1 hadoop-2.4.1-src.tar.gz hive-0.12.0-bin.tar.gzhadoop-1.2.1-bin.tar.gz hadoop-2.4.1.tar.gz mysql-connector-java-5.1.6-bin.jar[jifengjifeng02 hadoop]$ cp mysql-connector-java-5.1.6-bin.jar hive-0.12.0-bin/lib2:修改$HIVE_HOME/conf/hive-site.xmljavax.jdo.option.ConnectionURLjdbc:mysql://jifengsql:3306/hive?createDatabaseIfNotExisttrueJDBC connect string for a JDBC metastorejavax.jdo.option.ConnectionDriverNamecom.mysql.jdbc.DriverDriver class name for a JDBC metastorejavax.jdo.PersistenceManagerFactoryClassorg.datanucleus.api.jdo.JDOPersistenceManagerFactoryclass implementing the jdo persistencejavax.jdo.option.DetachAllOnCommittruedetaches all objects from session so that they can be used after transaction is committeddescriptionjavax.jdo.option.NonTransactionalReadtruereads outside of transactionsjavax.jdo.option.ConnectionUserNamedssusername to use against metastore databasejavax.jdo.option.ConnectionPasswordjifengpassword to use against metastore database3:启动hive[jifengjifeng02 hive-0.12.0-bin]$ hiveLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.propertieshive show tables;FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClienthive quit;报错网上查询后说是没把 mysql的jar包mysql-connector-java-5.1.10-bin.jar放在hive安装目录的lib下把 mysql-connector-java-5.1.6-bin.jar 替换成mysql-connector-java-5.1.10-bin.jar还是不行。检查 mysql发现连接不上换个虚拟机上mysql[dsslocalhost ~]$ mysql -u root -pEnter password:--root 登陆mysqlWelcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 70Server version: 5.6.16 MySQL Community Server (GPL)Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type help; or \h for help. Type \c to clear the current input statement.mysql create database hive;Query OK, 1 row affected (0.01 sec) --创建hive库mysql GRANT all ON hive.* TO dss% IDENTIFIED BY abc123;Query OK, 0 rows affected (0.03 sec)--给dss用户对hive库授权mysql flush privileges;Query OK, 0 rows affected (0.02 sec)--刷新系统权限表mysql set globalbinlog_formatMIXED;ERROR 1193 (HY000): Unknown system variable globalbinlog_formatmysql alter database hive character set latin1 ;Query OK, 1 row affected (0.00 sec)--更新字符集再次启动[jifengjifeng02 hive-0.12.0-bin]$ hiveLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.propertieshive show tables;OKTime taken: 6.273 secondshive没有错误了。4.Hive的运行模式即任务的执行环境1启动hive 命令行模式1直接输入#hive的执行程序2或者输入 #hive --service clil分为本地与集群两种我们可以通过mapred.job.tracker 来指明设置方式hive SET mapred.job.trackerlocal2.hive验证启动的方法l1、hive web界面的(端口号9999) 启动方式#hive --service hwi[jifengjifeng02 hive-0.12.0-bin]$ hive --service cliLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.propertieshive quit;[jifengjifeng02 hive-0.12.0-bin]$ hive --service hwi15/08/17 15:17:10 INFO hwi.HWIServer: HWI is starting up15/08/17 15:17:10 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog15/08/17 15:17:10 INFO mortbay.log: jetty-6.1.2615/08/17 15:17:10 INFO mortbay.log: Extract /home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-hwi-0.12.0.war to /tmp/Jetty_0_0_0_0_9999_hive.hwi.0.12.0.war__hwi__ow27i/webapp15/08/17 15:17:11 INFO mortbay.log: Started SocketConnector0.0.0.0:9999用于通过浏览器来访问hivehttp://jifeng02:9999/hwi/l2、hive 远程服务(端口号10000) 启动方式#hive --service hiveserver本条技术文章来源于互联网如果无意侵犯您的权益请点击此处反馈版权投诉本文系统来源php中文网
