hive-installation-in-ubuntu

Hive Setup
Download Hive Download and extract hive to a directory. Add below values to the .bashrc file export HIVE_HOME=/home/mkm/mm/softwares/hive-3.1.3 export PATH=$PATH:$HIVE_HOME/bin
How to Setup Hive on Ubuntu in 3 different ways. Embedded Metastore(Derby) Pre-requisite - Install Derby DB Local Metastore(Same Machine/JVM) Pre-requisite - Install MySQL Server Remote Metastore(Remote Machine/JVM)
Embedded Metastore(Derby) Will Update soon
Local Metastore(Same Machine/JVM) 1. Check MySQL Server Instance is running or not $ ps -e | grep mysql or sudo service mysql status If not, Install MySQL Server by following page in Pre-requisite section 2. Login to the MySQL server $ mysql -u root -p Create Hive Metastore Database mysql> CREATE DATABASE metastore_db; Create new user login for Hive mysql> CREATE USER 'hiveadmin'@'localhost' IDENTIFIED BY 'H1^e@dm!N'; Create Hive metastore tables mysql> GRANT all privileges on metastore_db.* to 'hiveadmin'@localhost; mysql> flush privileges 3. Install MySQL Connector JDBC/ODBC Driver for Ubuntu platform Download deb file as per installed Ubuntu version and install $ sudo dpkg -i directory_path/mysql-connector-xxxxxx-xxx.deb Search for installed mysql connector jar on file system. Normally it gets installed in /usr/share/java directory. Copy connector jar file to /home/mkm/mm/softwares/hive-3.1.3/lib or else you can mention Jar file path to below variable in HIVE_HOME/conf/hive-env.sh export HIVE_AUX_JARS_PATH= path_to_connector_jar/mysql-connector-xxxx.jar Alternatively we can mention Hadoop installation folder in the same file if not specified in .bashrc HADOOP_HOME=path-to-hadoop/hadoop 4. Configure Hive Setup Files Rename hive-defaults.xml to hive-site.xml Edit hive-site.xml and Change the values of below properties as per your MySQL & ODBC Driver Installation hive.metastore.db.type - mysql javax.jdo.option.ConnectionURL - jdbc:mysql://localhost:3306/metastore_db?createDatabaseIfNotExist=true javax.jdo.option.ConnectionUserName - hiveadmin javax.jdo.option.ConnectionPassword - H1^e@dm!N javax.jdo.option.ConnectionDriverName - com.mysql.cj.jdbc.Driver hive.exec.local.scratchdir - ${system:java.io.tmpdir}/${system:user.name} or /tmp/${user.name}
Remote Metastore(Remote Machine/JVM) Get the remote metastore details Edit hive-site.xml and Change the values of below properties as per your MySQL & ODBC Driver Installation hive.metastore.db.type - remote db type javax.jdo.option.ConnectionURL - jdbc:remote db type://remote host:port/metastore_db javax.jdo.option.ConnectionUserName - provided username javax.jdo.option.ConnectionPassword - provided authentication
Start the Hive Open a terminal and type hive $ hive hive> a hive cli will open
Troubleshoot Hive Installation If Hive did not started or not showing any databases/tables on select 1. Check the JDO(Java Data Object) details provided in hive-site.xml Verify the values of hive.metastore.db.type, javax.jdo.option.ConnectionURL, javax.jdo.option.ConnectionUserName, javax.jdo.option.ConnectionPassword, and javax.jdo.option.ConnectionDriverName 2. Directory/File permission issues Verify permission issue on the directories mentioned in hive-site.xml 3. If auto db create is not mentioned for the below property or unable to list databases/tables in hive console javax.jdo.option.ConnectionURL - jdbc:mysql://localhost:3306/metastore_db?createDatabaseIfNotExist=true we have to explicitly create the metastore db by below hive util $ schematool -dbType mysql -initSchema you can verify the metastore by below util $ schematool -dbType mysql -info 4. Ensure connector/J or required driver jar file on accessible path or classpath a. Copy the required jar file in HIVE_HOME/lib directory b. or the keep the driver file on classpath $ gedit .bashrc export PATH=$PATH:path_to_driver\connector-j-xxx.jar or else configure HIVE_HOME/conf/hive-env.sh file property export HIVE_AUX_JARS_PATH=path_to_jar_files
Compiled on SATURDAY, 03-AUGUST-2024, 08:12:26 AM IST