Creating Hive Indices

Vote count:

0

I am running a two node hadoop cluster running hive. I was able to increase performance by storing the tables as compressed sequence files. I am now trying to use indices to increase join performance. Table1 has 12 million rows, 1GB uncompressed, 160MB compressed. My question is, is it possible to add an index to this table on the join key. The first time I tried creating the index, it was almost 1GB in size. Is it possible to store the index as a compressed sequence file? Also, Once I have the index made do I need to query from the index directly or will hive know to use the index with the join? I saw 'set hive.optimize.autoindex=true;', not entirely sure how to use it. Thanks.

asked 46 secs ago

SailorLeroy

13