大数据分析工具Hue安装及集成各种组件
1、Hue安装
1.1、下载安装包
Hue的安装支持多种方式,包括rpm包的方式进行安装、tar.gz包的方式进行安装以及cloudera manager的方式来进行安装等,我们这里使用tar.gz包的方式来进行安装
Hue的压缩包的下载地址:
http://archive.cloudera.com/cdh5/cdh/5/
我们这里使用的是CDH5.14.0这个对应的版本,具体下载地址为
http://archive.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.14.0.tar.gz
通过百度网盘分享的文件:hue-3.9.0-cdh5.14.0.tar.gz
链接:https://pan.baidu.com/s/1GDNVLAvF_cs7vwLhmRKL7w
提取码:1i4y
1.2、使用WINSC 上传文件至/opt/tools,使用CP命令复制到/opt/sever
再使用tar -zxf hue-3.9.0-cdh5.14.0.tar.gz 进行解压
[root@zhang server]# ls
flume hadoop-2.8.0 hive-3.1.2 hue-3.9.0-cdh5.14.0
1.3、安装各种依赖包
yum install -y asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make openldap-devel python-devel sqlite-devel gmp-devel
1.4、Hue初始化配置文件
[root@zhang conf]# pwd
/opt/server/hue-3.9.0-cdh5.14.0/desktop/conf
[root@zhang conf]# vi hue.ini
[desktop]
# Set this to a random string, the longer the better.
# This is used for secure hashing in the session store.
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
# Execute this script to produce the Django secret key. This will be used when
# 'secret_key' is not set.
## secret_key_script=
# Webserver listens on this address and port
http_host=zhang
http_port=8888
# Choose whether to enable the new Hue 4 interface.
## is_hue_4=true
# Choose whether to still allow users to enable the old Hue 3 interface.
## disable_hue_3=false
# A comma-separated list of available Hue load balancers
## hue_load_balancer=
# Time zone name
time_zone=America/Los_Angeles
# Enable or disable Django debug mode.
django_debug_mode=false
# Enable development mode, where notably static files are not cached.
## dev=false
# Enable or disable database debug mode.
## database_logging=false
# Whether to send debug messages from JavaScript to the server logs.
## send_dbug_messages=false
# Enable or disable backtrace for server error
http_500_debug_mode=false
# Enable or disable memory profiling.
## memory_profiler=false
# Server email for internal error messages
## django_server_email='hue@localhost.localdomain'
# Email backend
## django_email_backend=django.core.mail.backends.smtp.EmailBackend
# Webserver runs as this user
server_user=root
server_group=root
# This should be the Hue admin and proxy user
default_user=root
# This should be the hadoop cluster admin
## default_hdfs_superuser=root
[[database]]
# Database engine is typically one of:
# postgresql_psycopg2, mysql, sqlite3 or oracle.
#
# Note that for sqlite3, 'name', below is a path to the filename. For other backends, it is the database name
# Note for Oracle, options={"threaded":true} must be set in order to avoid crashes.
# Note for Oracle, you can use the Oracle Service Name by setting "host=" and "port=" and then "name=<host>:<port>/<service_name>".
# Note for MariaDB use the 'mysql' engine.
engine=mysql
host=zhang
port=3306
user=root
password=root
# conn_max_age option to make database connection persistent value in seconds
# https://docs.djangoproject.com/en/1.9/ref/databases/#persistent-connections
## conn_max_age=0
# Execute this script to produce the database password. This will be used when 'password' is not set.
## password_script=/path/script
name=mysql
#options={"threaded":true}
# Database schema, to be used only when public schema is revoked in postgres
## schema=public
# Configuration options for specifying the Desktop session.
# For more info, see https://docs.djangoproject.com/en/1.4/topics/http/sessions/
# ------------------------------------------------------------------------
1.5、创建mysql中Hue使用的DB
create database hue default character set utf8 default collate utf8_general_ci;
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.01 sec)
mysql>
1.6、编译Hue
[root@zhang hue-3.9.0-cdh5.14.0]# make apps
编译成功之后,会在hue数据库中创建许多初始化表
2、启动Hue
前台启动:
./build/env/bin/supervisor
[root@zhang hue-3.9.0-cdh5.14.0]# cd /opt/server/hue-3.9.0-cdh5.14.0/
root@zhang hue-3.9.0-cdh5.14.0]# ./build/env/bin/supervisor
Traceback (most recent call last):
File "./build/env/bin/supervisor", line 9, in <module>
load_entry_point('desktop==3.9.0', 'console_scripts', 'supervisor')()
File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/supervisor.py", line 319, in main
setup_user_info()
File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/supervisor.py", line 257, in setup_user_info
desktop.lib.daemon_utils.get_uid_gid(SETUID_USER, SETGID_GROUP)
File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/lib/daemon_utils.py", line 45, in get_uid_gid
raise KeyError("Couldn't get user id for user %s" % (username,))
KeyError: "Couldn't get user id for user hue"
[root@zhang hue-3.9.0-cdh5.14.0]#
[root@zhang hue-3.9.0-cdh5.14.0]# su zhang
[zhang@zhang hue-3.9.0-cdh5.14.0]$ ./build/env/bin/supervisor
[INFO] Not running as root, skipping privilege drop
starting server with options:
{'daemonize': False,
'host': 'zhang',
'pidfile': None,
'port': 8888,
'server_group': 'root',
'server_name': 'localhost',
'server_user': 'root',
'ssl_certificate': None,
'ssl_certificate_chain': None,
'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
'ssl_private_key': None,
'threads': 50,
'workdir': None}
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission denied: 'yacctab.py'
3、WEB界面访问
地址:http://192.168.66.128:8888/hue/editor?editor=185

4、集成Hive
4.1、如果需要配置hue与hive的集成,我们需要启动hive的metastore服务以及hiveserver2服务(impala需要hive的metastore服务,hue需要hive的hiveserver2服务)
zhang@zhang bin]$ ./hive --service hiveserver2 &
[1] 5411
[zhang@zhang bin]$ which: no hbase in (/opt/server/hive-3.1.2/bin:/opt/server/flume/bin:/usr/lib/jvm/jdk1.8.0_65/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/lib/jvm/jdk1.8.0_65/bin:/opt/server/hadoop-2.8.0/bin:/opt/server/hadoop-2.8.0/sbin:/home/zhang/.local/bin:/home/zhang/bin)
2024-07-30 02:09:15: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/server/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/server/hadoop-2.8.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 8fe5cc1a-82a5-4227-9b6b-6f85d3c2c0ed
Hive Session ID = f77969f0-78fa-4de1-a881-abc12454bc5d
Hive Session ID = 2eea9353-057d-48cd-9fe6-04880cd8b298
Hive Session ID = 99227d48-0ebe-4d32-8514-a30894cd9012
OK
OK
OK
OK
OK
OK
OK
OK
具体可查看安装Hadoop组件Hive,配置Oracle存储元数据
4.2、修改Hue.ini
997 [beeswax]
998
999 # Host where HiveServer2 is running.
1000 # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
1001 hive_server_host=zhang
1002
1003 # Port where HiveServer2 Thrift server runs on.
1004 hive_server_port=10000
1005
1006 # Hive configuration directory, where hive-site.xml is located
1007 hive_conf_dir=/opt/server/hive-3.1.2/conf/
1008
1009 # Timeout in seconds for thrift calls to Hive service
1010 server_conn_timeout=120
1011
1012 # Choose whether to use the old GetLog() thrift call from before Hive 0.14 to retrieve the logs.
1013 # If false, use the FetchResults() thrift call from Hive 1.0 or more instead.
1014 ## use_get_log_api=false
1015
1016 # Limit the number of partitions that can be listed.
1017 ## list_partitions_limit=10000
1018
1019 # The maximum number of partitions that will be included in the SELECT * LIMIT sample query for partitioned tables.
1020 ## query_partitions_limit=10
1021
1022 # A limit to the number of rows that can be downloaded from a query before it is truncated.
1023 # A value of -1 means there will be no limit.
1024 ## download_row_limit=100000
1040 # Override the default desktop username and password of the hue user used for authentications with other services.
1041 # e.g. Used for LDAP/PAM pass-through authentication.
1042 auth_username=root
1043 auth_password=root
5、启动Hue访问Hive
./build/env/bin/supervisor
Web访问:http://192.168.66.128:8888/hue/jobbrowser#!jobs

更多推荐
所有评论(0)