Using Visual Studio to Work With Hive Scripts

I was building some HDInsight Azure Data Factory pipelines the other day and the Hive scripts were not doing what I wanted to them to do (although they were doing what was being asked).I needed to isolate the Hive scripts to see what they were doing at each stage of the script. I didn’t want to have to invoke the whole ADF just to look at the Hive section. I was in Visual Studio already so troubleshooting was easy from here. Here’s how I solved it.

Open up Server Explorer

Connect to your Azure Subscription

Navigate to the HDInsight node

Expand your clusters

Right click on the cluster of choice and select “Write a Hive Query”

Paste/Write your query

Another quirk of my Hive Script was that I was passing in values from the activity to the Hive Script itself. I knew what the values were that I was using but didn’t want to change the script to use hard coded values instead of the reference to the config. Here’s what I mean

I wanted to pass in my values as parameters to the script in Visual Studio. No problem. Once you have finished writing your query

Hit Submit | Advanced

Now you can enter your configuration key value pairs

This for me was a really quick and useful way of troubleshooting my Hive scripts. Hope you find it useful.